Job Title: Data Engineer (4–8 Years)
Location: Pune | Bangalore | Chennai
Employment Type: Full-time
Work Mode: Work from Office (3 days/week at client office – mandatory)
Availability: Immediate joiners only
About the Role:
We are looking for experienced Data Engineers with 4 to 8 years of experience to join our team. You will design and build scalable data pipelines on AWS using Databricks, Apache Airflow, and Apache Iceberg. This role involves working on enterprise-grade data platforms with a focus on performance, governance, and reliability.
Note: A detailed background verification (BGV) will be conducted. A valid LinkedIn profile is mandatory for application consideration.
Key Responsibilities
- Design, develop, and optimize ETL/ELT pipelines using Databricks, PySpark, and Apache Airflow.
- Implement data lakehouse architectures using Apache Iceberg for ACID transactions, time travel, and schema evolution.
- Build and maintain cloud-native data solutions on AWS using services like S3, Glue, Lambda, EventBridge, SNS, SQS, and Lake Formation.
- Implement fine-grained data access controls with AWS Lake Formation.
- Develop event-driven and serverless data workflows for scalability and cost efficiency.
- Collaborate with data scientists, analysts, and software engineers to deliver high-quality data products.
- Senior-level candidates will also lead technical initiatives and mentor junior team members.
Required Skills
Must-Have for All Candidates:
- Databricks
- Apache Airflow
- Python and PySpark
- Apache Iceberg
- AWS services: S3, Glue/Crawler, Lambda, Lake Formation
Additional Skills Based on Experience:
- For 4–6 Years Experience:
- Hands-on experience with EventBridge, SNS, SQS, and Kafka. Terraform knowledge is a bonus.
- For 6–8 Years Experience:
- Advanced proficiency in Databricks, Airflow, Python, Iceberg, and core AWS services (S3, Glue, Lambda, Lake Formation). Ability to lead technical design and mentor team members.
Preferred Qualifications
- Experience with Terraform for infrastructure as code.
- Familiarity with Kafka or other streaming platforms.
…