About The Position
TGE is looking for a data engineer to join the team. As a data engineer, you will team up with the rest of our data team to design, develop and maintain multi-purpose, complex data systems and services for the company’ internal use. Work with cutting-edge technology for data processing and analysis for big data environments.
· Design, develop and maintain our cutting-edge Big Data frameworks. Including ETL, Click streams and other solutions
· Maintain a high standard of code (unit tests, technical documentation, 0 exceptions)
· Help define and design our current and future Big Data architecture.
· Work closely with our Data Integrity testers to ensure deliverables are accurate and timely.
· Work closely with the DataOps team to design deploy flawless CI/CD.
· Perform code reviews with the team to ensure excellence and quality of delivered code.
· 2 years of experience in Python/Scala/Node.Js or any other functional programming languages.
· Familiarity with relational (SQLServer/MySql/Postgres/etc) and non-relational DBs (MongoDB/HBase/Redis/etc.)
· Experience in developing, testing, and maintaining ETL/ELT processes.
· Experience with the Hadoop ecosystem (HDFS, Yarn, etc.)
· Experience working with Spark.
· Experience working with Kafka.
· Experience with relational (Redshift/SQL Server/MySql/etc.) and non-relational DBs
· Experience with AWS.
· Basic knowledge of pipelines/data orchestration
· Experience in any Event Store management system