Kategória: Munka, állás
Honlap:
Részletek...The Wow
This role will contribute in designing and developing our platform to fulfil the business needs and to also improve our systems. Within this capacity you will have the chance to design new data pipelines, maintain platforms hosted on data streams for batch and real-time loading and real-time visualisations.
A taster of what you will be involved with:
Maintaining and evolving our existing data platform;
Building processes to ingest data from Kafka, APIs and Databases using AWS MSK Connect.
Development and Maintenance of real-time data processing applications using frameworks like Spark Structured Streaming and Kafka Streams.
Applying transformations to data streams;
Being involved in data modelling following standards e.g. Inmon, Kimball, Data Vault;
Ensuring data quality by verifying data consistency and accuracy;
Keeping up to date on research and development of new technologies and techniques to enhance our data platform;
Have an investigative mindset to be able to troubleshoot - thinking outside the box when it comes to troubleshooting problems and incident management;
Full ownership on projects and tasks assigned together with being able to work within a Team;
Able to document well processes and perform knowledge sharing sessions
What we are looking for:
Must haves:
Experience with modern cloud database technologies such as Snowflake ;
Experience with orchestrating data pipelines using Airflow ;
Experience with AWS Glue ;
Experience with Apache Iceberg;
Experience with SQL , and Data Integration Tools
Experience with any programming language like Python or Scala;
Kno
