Data Engineer

Responsibilities

• Responsible for setting up a scalable DataWarehouse and building data pipeline mechanisms to integrate the data from various sources for all of Klub s data.
• Setup data as a service to expose the needed data as part of Apis.
• Have a good understanding on how the finance data works.
• Standardise and optimise design thinking across the technology team.
• Collaborate with stakeholders across engineering teams to come up with short and longterm architecture decisions.
• Build robust data models that will help to support various reporting requirements for the business ops and the leadership team.
• Participate in peer reviews provide code/design comments.
• Own the problem and deliver to success.

Requirements

• Prior experience on Backend and Data Engineering systems.
• Should have at least 2.5 years of working experience in distributed systems or 2.5 years of experience in Data Engineering.
• Deep understanding on python tech stack with the libraries like Flask scipy numpy pytest frameworks.
• Good understanding of Apache Airflow or similar orchestration tools.
• Good knowledge on data warehouse technologies like Apache Hive or similar.
• Good knowledge on Apache PySpark or similar.
• Good knowledge on how to build analytics services on the data for different reporting and BI needs.
• Good knowledge on data pipeline/ETL tools.
• Good knowledge on Trino / graphQL or similar query engine technologies.
• Deep understanding of concepts on Dimensional Data Models.
• Familiarity with RDBMS (MySQL/ PostgreSQL) NoSQL (MongoDB/DynamoDB) databases & caching(redis or similar).
• Should be proficient in writing sql queries.
• Good knowledge on kafka.
• Be able to write clean maintainable code.

Nice to have

• Built a Data Warehouse from the scratch and set up a scalable data infrastructure.
• Prior experience in fintech would be a plus.
• Prior experience on data modelling.
• Prior experience working with open source data engineering tools.

data,data engineering,data warehouse,data models,pipeline,reporting,design,models