Senior Data Engineer
Duration: 3 Months
Location: Remote (USA)
Hands-on Senior Data Engineer to support enhancement, stabilization, and delivery of a large-scale healthcare data lake built on Azure Databricks. The role will drive development of scalable data pipelines, platform improvements, production issue remediation, and onboarding of new data products supporting interoperability, analytics, and regulatory use cases.
Key Responsibilities
Design, develop, and optimize data pipelines using Azure Databricks, PySpark, SQL, and Delta Lake
Build and enhance Medallion Architecture Bronze and Silver layer data products
Support platform enhancements, defect remediation, and production stabilization activities
Collaborate with architects, product owners, QA, and business stakeholders to define and deliver solutions
Perform root cause analysis and drive resolution of technical and data-related issues
Support release planning, deployment activities, and production readiness
Required Skills
5+ years of Data Engineering experience.
Strong expertise in Azure Databricks, PySpark, SQL, ADLS, and Delta Lake
Knowledge of Azure Data Factory, Unity Catalog, CI/CD, and GIT
Experience with ETL/ELT development, data warehousing, and dimensional modeling
Strong troubleshooting, performance tuning, and debugging capabilities
Experience working in Agile delivery environments
Excellent communication and stakeholder management skills
Preferred Skills
Healthcare payer domain experience (Claims, Membership, Provider, Clinical and more)
Experience with FHIR/Interoperability data models
Experience supporting enterprise-scale cloud data platforms