Data Engineer Job Description

Are you a skilled Data Engineer looking for an exciting opportunity in Gain America inc, We have a position for you at Gain America Inc! As a Data Engineer, you will play a crucial role in designing and developing big data applications using cutting-edge technologies. You will have the chance to work with a talented team in an offshore model and contribute to the development of logical and physical data models for big data platforms.

Responsibilities:

• Design and develop big data applications using open source technologies.
• Automate workflows with Apache Airflow.
• Create data pipelines using Apache Hive, Apache Spark, and Apache Kafka.
• Provide maintenance and enhancements to existing systems.
• Mentor junior engineers and lead design reviews.
• Prioritize backlog using JIRA and act as a point of contact for the business domain.

Requirements:

• 4+ years of recent GCP experience.
• Experience in building data pipelines in GCP.
• Experience with data warehouse solutions and data products.
• Hands-on experience with Hadoop, Hive, Spark, Airflow, or similar technologies.
• Experience in modeling and designing schemas for data lakes or RDBMS platforms.
• Proficiency in Python, Java, and Scala.
• Ability to work with large datasets.
• Exposure to Agile development methodologies.

Preferred Skills:

• Experience with Gitflow, Atlassian products, and continuous integration tools.
• Knowledge of Kubernetes, Docker, and Jenkins.
• Domain experience in Retail is a plus.