Data Engineer

What to expect of your role:

We are looking for a Data Engineer to help us develop and maintain our data platform. Our infrastructure is primarily AWS-based, with Airflow, S3 and Redshift at its core, alongside a legacy stack based on MS SQL Server, SSIS and custom Python scripts. You will work on building reliable data pipelines, improving our data infrastructure, and gradually modernising legacy processes

You will be expected to:

• Design, develop and maintain scalable and reliable data pipelines and ETL/ELT processes;
• Develop and optimise data warehouse solutions;
• Develop and maintain data infrastructure, primarily on AWS, while also supporting parts of our legacy infrastructure;
• Develop APIs and data integrations;
• Contribute to the development and maintenance of real-time data pipelines using Kafka;
• Automate and optimise data processes and workflows;
• Monitor, troubleshoot and improve the reliability, performance, security and data quality of pipelines and infrastructure;
• Work closely with the analytics team and other stakeholders to understand their data needs and deliver effective solutions;
• Maintain clear and up-to-date documentation for data pipelines, infrastructure and workflows;
• Contribute to the improvement and modernisation of existing data processes and architecture.

What you need to succeed in this role:

• 2+ years of professional experience in data engineering;
• Strong SQL and Python skills;
• Hands-on experience with AWS, including S3, Redshift, Glue and Lambda;
• Hands-on experience with Apache Airflow;
• Strong understanding of data warehousing concepts and data modelling;
• Hands-on experience designing, developing and maintaining production data pipelines, with a focus on reliability, monitoring and data quality;
• Solid understanding of relational and non-relational databases;
• Familiarity with Git and CI/CD practices, preferably GitLab;
• Experience developing APIs and data integrations;
• Kafka and real-time data processing experience is a plus;
• MS SQL Server and SSIS experience is a plus;
• Good written and verbal communication skills;
• Upper-intermediate English or higher.