Description:
Job Description
Summary
Our client is seeking a highly experienced Senior Data Engineer to lead the design, development, and optimization of data pipelines, APIs, and data platforms. This role will focus on ETL/ELT processes using Matillion and Snowflake, API development, and integration with machine learning workflows and Databricks. The ideal candidate will have a strong background in data engineering, cloud platforms, and modern data architecture.
Responsibilities
Design, build, and maintain ETL/ELT pipelines using Matillion and Snowflake. Develop and manage RESTful APIs for data access and integration. Collaborate with data scientists and ML engineers to integrate data pipelines with machine learning workflows. Optimize Snowflake data warehouse performance and manage data models. Implement data quality, governance, and security best practices. Work with Databricks for data processing, transformation, and ML model support. Automate data validation and reconciliation processes. Document data architecture, pipelines, and integration processes.
Qualifications
Bachelors or Masters degree in Computer Science, Engineering, or a related field. 8+ years of experience in a data modeling, data warehousing, and data integration role.
Required Skills
Strong experience with Matillion ETL and Snowflake. Proficiency in Databricks Experience in data management requirements, integration platforms, and have expertise with APIs. Experience with API development using frameworks. Familiarity with Databricks and integrating Machine Learning workflows. Solid understanding of data warehousing, data modeling, and ELT best practices. Experience with CI/CD pipelines, version control (Git), and DevOps practices. Strong proficiency in SQL and at least one programming language (e.g., Python, Java, or Scala). Experience with data modeling, data warehousing, and data integration. Familiarity with cloud platforms and data services. Understanding of data governance, security, and compliance requirements
02 Jul 2025;
from:
gumtree.co.za