Data Engineer – Databricks Expert

📍 Remote (India-based) | 🕒 Full-time | 🎯 Immediate Joiners Preferred

Are you passionate about building scalable data pipelines and optimizing performance with cutting-edge tools like Databricks and Apache Spark? We're looking for a seasoned Data Engineer to join our team and work on high-impact data products for a leading global client.

🔧 Tech Stack & Skills

5+ years in Python (incl. Pytest for testing)
7+ years of SQL & Query Optimization
3+ years of API development
Hands-on with Databricks CLI, DLT Pipelines, Unity Catalog, and Delta Lake
Strong grasp of Apache Spark, Structured Streaming, Medallion Architecture
Familiarity with Databricks Workflows, Asset Bundles, and REST APIs
Solid understanding of CDC (Debezium preferred)
Exposure to AWS, DevOps, CI/CD, and Git
Experience with SQL Server, Data Warehousing, Semi-Structured Data, and MS SSIS

📌 Key Responsibilities

Develop and deploy robust DLT pipelines, workflows, and ingestion tasks
Maintain production streaming and batch jobs with tight SLAs
Monitor pipelines daily, troubleshoot failures, and document resolutions
Analyze data lineage using codebase and Databricks tools
Collaborate to automate data quality checks
Write clean code, conduct PRs, testing, and deployments
Communicate clearly on incidents, blockers, and task progress

💼 What You Bring

Strong problem-solving mindset
Passion for clean, scalable, and well-documented code
Ability to work independently and in a fast-paced environment
Comfortable collaborating across time zones

Data Engineer - Databricks Expert