Senior data mühəndisi

Vakansiya haqqında

We are looking for a hands-on Leading / Senior Data Engineer to own critical ingestion and lakehouse workflows in a high-throughput, production-grade data platform.

Vəzifə öhdəlikləri

Designing a data lake to process and store an array of (un)structured data loaded from raw sources. Development of the data model - properly store the data and access it as needed for business purposes
Development of the integration process - development of integration with various systems so that they can have a single view of key indicators when making decisions
Data preparation and ETL - development of a pipeline for extracting, transforming and loading data Developing and monitoring data pipelines to ensure data quality and integrity
Automation and optimization of the data transformation process
Working closely with other teams in the organization to ensure data is available and accessible for analytics and decision-making

Tələblər

Degree in computer science, information science, engineering, mathematics, or related technical discipline
5+ years experience with SQL and NoSQL technologies
Strong experience working with Oracle and PostgreSQL databases
Extensive experience in ETL development, with strong skills in Python programming
Advanced knowledge of PySpark for large-scale distributed data processing
Proficient in Airflow for scheduling, orchestrating and monitoring complex ETL/ELT workflows
Expertise in Kafka for event streaming and messaging pipelines
Experience working with MinIO (S3-compatible storage), managing data in Parquet and Iceberg formats
Familiarity with Lakehouse concepts
Deep understanding of MPP architecture concepts

The following experience will be considered as an advantage:

Familiarity with Impala for interactive SQL querying on Big Data
Experience with Greenplum
Familiarity with containerization (Docker, Kubernetes)
Experience with CDC tools like Debezium and Oracle GoldenGate for real-time data replication
Experience with data visualization tools (e.g. Tableau, PowerBI, etc.)
Understanding of DataOps and DevOps principles
Knowledge of data modeling and database design
Experience with data engineering best practices, such as data security, data access control and data governance
Knowledge of Grafana for monitoring metrics
Proficient with GitLab for version control
Experience mentoring junior data engineers and collaborating with cross-functional teams (Data Scientists, Analysts, DevOps)