[Remote] Lead Engineer, Data Processing (Databricks)

Note: The job is a remote job and is open to candidates in USA. CargoSprint is a company dedicated to transforming the cargo industry through innovative digital solutions. The Lead Engineer, Data Processing will oversee the Data Processing team, managing the design and implementation of data pipelines while ensuring operational reliability and data quality. Responsibilities • Lead the Data Processing team: technical direction, delivery cadence, coaching, and hiring support (as needed) • Own core processing systems end-to-end: design, implementation, quality, observability, and on-call readiness • Drive modernization of legacy processing: simplify flows, reduce failure modes, improve performance and cost • Build and operate pipelines (batch and/or streaming) with strong data quality, lineage, and backfill strategies • Establish SLAs/SLOs for key pipelines and improve monitoring, alerting, and incident response • Partner with application engineering, finance, and analytics to prioritize work and deliver dependable outputs • Raise the engineering bar via standards, PR reviews, runbooks, and pragmatic architecture decisions Skills • 8+ years building production systems; 2+ years leading engineers or acting as a team tech lead • Strong experience with data processing systems (ETL/ELT, streaming, batch processing, orchestration) • Hands-on experience with Databricks and Spark (PySpark/Scala) and strong SQL • Experience modernizing legacy systems safely and incrementally • Operational discipline: monitoring, alerting, incident response, and root cause analysis for pipelines • Ability to drive cross-team alignment and translate requirements into reliable delivery • Lakehouse patterns (Delta Lake, Iceberg, or Hudi) and data catalog/governance tools • .Net or Java expertise • Event-driven architectures (Kafka/Kinesis/PubSub) and streaming pipelines • FinTech, payments, billing, invoicing, or other high-volume transactional domains • Experience enabling self-serve analytics for business users • Logistics, cargo, or supply chain experience • Spanish language proficiency Benefits • Health and Wellness: Medical, dental, and vision plans for you and your family • Future-Ready: 401(k) with company match • Work Life Balance: Generous flexible PTO program and paid holidays • Grow With Us: Professional development opportunities Company Overview • CargoSprint focused in enhancing human experience in the cargo industry, through digital solutions. It was founded in 2012, and is headquartered in Peachtree City, Georgia, USA, with a workforce of 201-500 employees. Its website is https://cargosprint.com/. Company H1B Sponsorship • CargoSprint has a track record of offering H1B sponsorships, with 2 in 2025, 1 in 2024, 1 in 2023. Please note that this does not guarantee sponsorship for this specific role. Apply tot his job Apply To this Job