Event-Driven Order Pipeline
PLACEHOLDER — Replaced a cron-polling order processor with a Kafka pipeline. Cut end-to-end latency from ~4min to under 900ms.
- Stack
- Node.js · TypeScript · Kafka · PostgreSQL · Docker
- Links
- GitHub
DUMMY CONTENT. Structure is real, words are placeholder. Replace before launch.
The problem
Placeholder. A cron job polled an orders table every five minutes and pushed rows downstream. It double-processed on overlapping runs and fell behind under load.
What I built
Placeholder. A Kafka-based pipeline using the transactional outbox pattern, with idempotent consumers keyed on order ID.
Architecture
Placeholder — architecture diagram goes here (SVG, theme-aware, real text).
Three decisions that mattered: outbox over dual-write, consumer groups partitioned by customer ID, and dead-letter topic with replay.
Results
Placeholder numbers. End-to-end latency ~4min to under 900ms. Zero duplicate orders across 30 days.
What I'd do differently
Placeholder. Partitioning by customer ID created hot partitions for the largest accounts. A composite key would have distributed better.