Software Engineer & Software Architect
13112023.1
Your Tasks: - Develop, optimise & operate data processing pipelines using Python-based landscape (PySpark), including streaming data processing technologies (Kafka and Spark Streaming)
- Design and improve cloud-based solutions and integrate them with on-premise environments
- Collaborate with teammates on the architectural aspects of solution design
- Implement software development best practices and standards within the team's development process
- Provide support and mentoring to teammates in software development
Your Core Competencies:- Minimum of 2 years of practical experience in Databricks development
- Hands-on experience with Apache Spark (RDD, dataframes, streaming)
- SQL knowledge for developing and optimizing complex analytical queries
- Experience with message broker technologies: Kafka, Rabbit MQ, Active MQ, or similar
- Practical experience in setting up and configuring CI/CD pipelines (e.g., Github, Github actions, Artifactory)
- Willingness to mentor less experienced colleagues in code development through pair programming, code review, and sharing software design patterns
- Experience as a software developer in data engineering projects, including streaming data processing, batch processing, ETL/ELT, pipelines orchestration, Hadoop, NoSQL, DWH, OLAP, etc.
- Hands-on experience with AWS cloud computing (e.g., EC2/AWS based Linux, S3, AWS Lambda)