Job Description

We are hiring for a Lead Data Engineer. The role focuses on production support and management of data engineering pipelines, including data ingestion, processing, monitoring, and incident management.
A strong fit will have 8+ years of production data engineering experience with GCP, CDC pipelines, Java, and data processing technologies.

What you bring

• 8+ years of experience supporting and managing data engineering pipelines in production environments.
• Strong experience with Change Data Capture using Debezium and file-based ingestion using CSV.
• Hands-on experience with GCP Pub/Sub, Dataflow (Apache Beam), and Datastream.
• Experience working with Cloud SQL (PostgreSQL) environments.
• Experience managing staging and curated data layers for downstream consumption.
• Strong understanding of production monitoring, alerting, and incident management processes.
• Experience with API-based integrations within data pipelines.
• Strong Java programming experience for data processing and support activities.

What you'll do

• Provide end-to-end production support for data ingestion and processing pipelines.
• Monitor and manage CDC pipelines and batch ingestion processes for multiple source systems.
• Handle incidents, perform root cause analysis, and implement preventive fixes.
• Work with GCP Pub/Sub topics, subscriptions, DLQs, and alerting mechanisms.
• Support staging and transformed data processing layers.
• Perform production deployments and coordinate testing across lower environments.
• Troubleshoot and resolve data, performance, and pipeline failures within SLAs.
• Serve as technical lead for AMS support and coordinate day-to-day operations.
• Own incident communication, escalation, and stakeholder updates.
• Maintain operational runbooks and provide guidance to L2/L3 support teams.
• Identify recurring issues and drive continuous improvements for system stability and reliability.

Job Tags

Similar Jobs