We are looking for data engineers who take ownership of the design and implementation of data storage and processing infrastructure to join an innovative, young yet world famous data & software company. Our clients bring mission-critical insights to customers around the world by collecting & analysing vast quantities of data and identifying patterns and connections within that data.
Your core responsibilities will include you facilitating levelling up many aspects of their business. You will empower clients with more insights from better data. You will ensure that the data is acquired can be utilised to the fullest by various stakeholders. This includes their end-user application, and also their Data Scientists, marketing etc. You will lead the design and construction of a modern, scalable evolved data processing and storage infrastructure that fits our requirements like a glove. Your main tasks:
- Developing a modern and scalable data processing pipeline. This includes designing, building, optimising, scaling and maintaining it. You will implement some components yourself and delegate others to colleagues.
- Picking and deploying the appropriate data storage systems for our various requirements. You utilise the strengths of the various types of systems (SQL, NoSQL, key-value stores, document stores, graph databases, etc) to realise the best possible solution.
- Facilitating data acquisition and ingestion
- Closely collaborating with machine learning experts, data scientists and business developers, which each have their own requirements.
- Bsc/Msc in Software Engineering/Computer Science .
- Professional experience with building scalable data processing pipelines.
- Extensive knowledge of the various types of data storage systems available, together with their strenghts and weaknesses, and are able to optimally utilise them to achieve the best possible overall system architecture.
- You actively contribute your experience and opinions to help continuously improve our development processes, tools/stacks and methodologies.
- Several years of professional experience with Python.
- Experience with common development tools like Git and Docker.