Designing event-driven systems with message queues

Event-driven architecture lets services react to changes instead of polling for them. When a user signs up, an order is placed, or a file finishes processing, the system emits an event. Other services listen, react, and keep moving without waiting. This post covers the patterns, trade-offs, and operational details that make event-driven systems reliable at scale.

1. Events are facts, not commands

An event describes something that already happened: OrderCreated, PaymentSucceeded, FileUploaded. It is not a request. Producers emit events; consumers decide what to do. That separation is what gives you loose coupling.

Good event names are past tense, specific, and versioned:

• OrderCreated v1
• OrderCreated v2

Versioning lets you evolve the payload without breaking existing consumers.

2. Message queues as the backbone

A message queue decouples producers from consumers and buffers bursts:

• **Kafka** — high-throughput, durable log. Great for event sourcing and analytics pipelines.
• **RabbitMQ** — flexible routing with exchanges and queues. Great for task distribution and workflows.
• **AWS SQS / Google Pub/Sub** — managed, low ops. Great when you want the cloud provider to run the infra.

Choose based on throughput, durability needs, and operational preference.

3. Event sourcing vs. event notification

Not every event needs a full event store. Understand the difference:

• **Event notification** — the event tells consumers something changed; they fetch current state from their own source of truth. Lightweight, simple.
• **Event sourcing** — the event *is* the state. You rebuild current state by replaying events. Powerful for audit trails, debugging, and temporal queries, but adds complexity.

Most systems start with event notification and move to event sourcing only where the audit or replay value is worth the cost.

4. Idempotency and ordering

Consumers must tolerate:

• **Duplicate delivery** — queues retry. Make handlers idempotent by storing processed event IDs.
• **Out-of-order delivery** — partitions or delayed retries can reorder events. Use sequence numbers or causal ordering when order matters.

A simple idempotency pattern:

ts
const seen = new Set<string>();
async function handle(event: Message) {
  const id = event.headers["idempotency-key"];
  if (seen.has(id)) return;
  seen.add(id);
  await process(event);
}

5. Dead-letter queues and observability

Not every event will succeed. A dead-letter queue (DLQ) captures failures so you can inspect, retry, or alert without blocking the main pipeline.

Pair DLQs with:

• Structured logging on every handler.
• Metrics for throughput, latency, and failure rate.
• Alerts when DLQ depth exceeds a threshold.

If you cannot see failures, you cannot fix them.

6. A practical example

A checkout flow:

1. User places order → emit OrderCreated. 2. Inventory service consumes OrderCreated → reserve stock → emit StockReserved or StockFailed. 3. Payment service consumes StockReserved → charge card → emit PaymentSucceeded or PaymentFailed. 4. Notification service consumes PaymentSucceeded → send confirmation email.

Each service owns its data and reacts to events. No synchronous chain. If email is down, the order still completes.

Checklist

• [ ] Events are named, versioned, and documented.
• [ ] Producers and consumers are independently deployable.
• [ ] Consumers are idempotent.
• [ ] Ordering requirements are explicit per event type.
• [ ] DLQ + alerting is in place.
• [ ] End-to-end latency is measured and budgeted.

Event-driven systems trade determinism for resilience. When done carefully, they let teams move faster, fail safer, and scale cleaner than synchronous alternatives.