Senior Data Engineer - ETL & Pipelines

About the Client

Our client operates a high-volume B2C creator platform in the entertainment industry, serving more than 200,000 active users and 30,000 creators, with payment events flowing through the platform every second.

Data is one of the company’s most valuable competitive advantages. In this role, you will work closely with an experienced data analyst to transform a continuous flow of events into clean, reliable, and query-ready data that supports product, payments, and growth decisions.

The Role

We are looking for a Data Engineer who will take end-to-end ownership of the company’s ETL pipelines - the systems responsible for moving data from the transactional platform, including MongoDB, payment streams, and behavioral events, into the analytics environment where it can be turned into actionable insights.

This is not simply a dashboard integration role.

You will design and maintain pipelines that remain accurate, reliable, and timely while millions of events are processed and financial transactions occur every second.

Data integrity is critical. A missing payment event or duplicated transaction is not merely a technical issue—it can have a direct financial impact.

What You’ll Do

Own the pipelines: Design, build, and maintain ETL and streaming pipelines that ingest high-volume event and payment data and deliver it to the analytics warehouse reliably and accurately.

Model the data: Partner closely with the Lead Data Analyst to create warehouse schemas that answer essential business questions related to creator earnings, payment throughput, retention, lifetime value, and cohort behavior.

Engineer for failure: Build idempotent, replayable, and observable jobs that can withstand traffic spikes, retries, and partial failures without compromising downstream data.

Process high-volume events: Move data through queue infrastructure including BullMQ and Redis, AWS SQS, and RabbitMQ, while instrumenting pipelines through OpenTelemetry.

Protect data integrity: Implement reconciliation checks, deduplication, backfills, and reliable handling of late or out-of-order events.

Protect financial transactions: Collaborate with payments and platform teams to ensure that every transaction is captured once, accurately, and on time.

The Technology Stack

Languages: Node.js and TypeScript as the primary technologies, with some .NET and C#

Data stores: MongoDB as the primary transactional database and Redis

Queues and streaming: BullMQ, AWS SQS, and RabbitMQ

Analytics and warehouse: PostHog, its data warehouse layer, and SQL data modeling

Platform: AWS, including S3 and SNS, Docker, Kubernetes, and GitHub Actions

Observability: OpenTelemetry tracing across every pipeline

Requirements

About You

You are a strong engineer with substantial experience building high-volume data and ETL pipelines.

You will ideally bring:

Approximately 5–10 years of relevant engineering experience

Strong experience with batch and/or streaming data pipelines

Deep knowledge of event-driven architectures and message queues

A strong understanding of idempotency, at-least-once versus exactly-once processing, retries, and out-of-order event delivery

Professional experience with TypeScript and Node.js

Solid SQL and data-modeling capabilities

The ability to collaborate with analysts and design schemas that make business queries efficient and intuitive

A strong focus on data integrity, particularly when working with payments and financial transactions

Experience implementing reconciliation, auditing, monitoring, and data-quality controls

Previous experience in a high-volume B2C environment such as gaming, gambling, fintech, payments, or another real-money platform

Experience with .NET or C# would be considered an advantage.

Nice to Have

Experience working with payment or transaction data and financial reconciliation

Experience with MongoDB change streams or change data capture

Familiarity with data warehouse tooling and dbt-style modeling

Experience working with Kubernetes and infrastructure as code

About 2am.tech

We are 2am.tech, a team of builders and problem solvers united by a shared commitment to delivering excellence.

This commitment has shaped a company culture focused on creating solutions and products that consistently exceed expectations. Our drive to learn, evolve, and solve meaningful business challenges has taken us on a journey that goes #beyondsoftware.

As part of that journey, we partner with ambitious companies to help them identify and hire exceptional technology professionals.

We are currently managing the recruitment process for a client seeking an experienced Data Engineer — ETL & Pipelines to join its team.

This is a direct-hire opportunity. The successful candidate will be employed or engaged directly by the client, while 2am.tech will coordinate the recruitment and selection process.

Originally posted on Himalayas