Innomerc Choose your depth. Learn at your own pace.
Understand it
A message queue holds work until another component can process it. A producer submits a job; a consumer receives it and performs the work. This separates the moment a request arrives from the moment the job finishes.
For Cedar’s receipt emails, the checkout can enqueue a receipt job while a worker sends it later. A queue absorbs some bursts, but it does not create unlimited capacity. If jobs arrive faster than workers finish, the backlog grows.
Under the hood
Delivery and acknowledgement are separate. After a worker processes a message, it acknowledges it according to the broker’s protocol. If it fails before acknowledgement, the message may be delivered again. The worker therefore needs to handle duplicates safely.
Publisher confirmation and consumer acknowledgement cover different boundaries. Durability, persistence and replication settings also differ by product. An at-least-once delivery policy does not provide exactly-once effects in a separate database or payment system.
Practitioner perspective
Design idempotency around a stable operation identifier and an atomic record of completed effects. A crash after an effect but before acknowledgement is a classic duplicate window. Retries need limits and backoff; poison messages should move to a reviewable dead-letter path.
Monitor oldest-message age, queue depth, throughput, retry rate and consumer health. Ordering can change with parallel consumers and retries. Backpressure should protect upstream services instead of allowing an unbounded buffer to consume all resources.
Message Queues
A message queue holds work until another component can process it. A producer submits a job; a consumer receives it and performs the work. This separates the moment a request arrives from the moment the job finishes.
Illustrated analogy · Select the image to enlargeWatch it work
A fictional, simplified model. Select a component to inspect it, or follow the steps in order.
Messages move independently of workers
Watch delivery, acknowledgement, redelivery and load control.
Processing completion and broker acknowledgement are distinct steps.
How to use this example
Select Next step to read at your own pace, or Animate step to watch one exchange. Nothing advances until you choose.
What it means
Everyday example
State in this scenario
Inspect components & read every step
- Producer
Software that creates messages for another part of a system to process.
Example: A shop adds an order-confirmation task to a queue.
- Work queue
A place where tasks wait until a worker can process them.
Example: Confirmation emails wait in a queue during a busy sale.
- Worker
Software that takes a queued task and performs the work.
Example: An email worker sends an order confirmation.
- Dead-letter queue
A separate destination for messages that could not be processed under the configured retry policy.
Example: A repeatedly failing task is parked for investigation instead of retried forever.
- Worker B
A second worker processing tasks from the system.
Example: Another worker handles a delivered task; duplicate-safe processing can prevent repeating an effect.
Delivery + acknowledgement
Processing completion and broker acknowledgement are distinct steps.
- Producer Work queue: Job 42. A job is accepted under the broker’s publish policy.
- Work queue Worker: Deliver 42. Delivery transfers work to a consumer.
- Worker Work queue: ACK 42. After durable processing, the worker acknowledges.
Redelivery + deduplication
At-least-once delivery can repeat a message; idempotent processing prevents repeated effects.
- Producer Work queue: Job 42. The broker accepts the job under its configured publish policy.
- Work queue Worker: Deliver 42. The worker commits the effect but loses its acknowledgement.
- Work queue Worker B: Redeliver 42. After detecting failure, the broker redelivers.
- Worker B Work queue: ACK duplicate. The consumer recognizes the completed idempotency key.
Finite retries dead letter
A configured dead-letter policy isolates a repeatedly failing message.
- Producer Work queue: Poison job. A malformed job enters the queue.
- Work queue Worker: Attempt 1. The consumer cannot process the message.
- Worker Work queue: Retry policy. A bounded application policy permits another attempt.
- Work queue Worker: Last attempt. The finite retry budget is exhausted.
- Work queue Dead-letter queue: Dead-letter route. Configured routing sends the message for investigation.
Backpressure
Prefetch limits unacknowledged deliveries; producer throttling needs its own flow-control policy.
- Producer Work queue: Burst of jobs. Work arrives faster than this worker can finish.
- Work queue Worker: Prefetch: 1. Limit this consumer to one unacknowledged delivery.
- Work queue Producer: Flow-control signal. A configured capacity policy slows further publishing.
- Worker Work queue: ACK. Completion frees capacity for another delivery.
Competing consumers
A work queue distributes deliveries; it does not broadcast each job to every worker.
- Producer Work queue: Jobs A + B. Two distinct jobs enter the work queue.
- Work queue Worker: Job A. One available consumer receives A.
- Work queue Worker B: Job B. Another consumer receives B.
- Worker B Work queue: ACK B. Completion order can differ from publication order.
- Worker Work queue: ACK A. The other job completes later.
Fictional teaching trace. One moving dot represents an exchange, not a measured network capture.
Open the detailed lesson walkthrough
Explore the system
Producer
Checkout publishes a receipt job with a stable ID.
Read every step without animation
Successful job
- Producer. Checkout publishes a receipt job with a stable ID.
- Queue. The broker stores the pending work according to its configuration.
- Consumer. A worker processes the job.
- Acknowledge. The worker tells the broker this delivery is complete.
Completion and acknowledgement are explicit steps.
Crash before acknowledgement
- Deliver. The worker receives receipt job 42.
- Effect committed. The receipt operation is recorded as complete.
- Worker fails. The worker stops before acknowledging.
- Redelivery. The broker delivers job 42 again.
- Idempotency check. The next worker recognizes the completed operation and avoids repeating the effect.
Duplicate delivery does not have to mean duplicate business effects.
Explore deeper: additional experiment
What happens after a worker crashes?
The worker recognizes the committed idempotency key and acknowledges without repeating the effect.
A local teaching model with fictional inputs. It does not test a live system.
Attack view
An unbounded or repeated job can exhaust workers and delay legitimate work. The visual shows a failure retry rather than an attack against a real broker.
Defender view
- Authenticate producers and consumers and restrict queue permissions.
- Validate message payloads and cap retry attempts.
- Make effects idempotent and alert on growing backlog or repeated failures.
Common mistakes
- A delivered message is not necessarily a completed job.
- At-least-once delivery can produce duplicates.
Check your understanding
Active recall: explain this idea in 30 seconds
A message queue holds work until another component can process it. A producer submits a job; a consumer receives it and performs the work. This separates the moment a request arrives from the moment the job finishes.
Now give an example and name one limitation. Revisit the explanation if you get stuck.
Primary sources
Original explanations informed by these specifications and official resources. Source versions and lesson review are separate from product updates.
- RabbitMQ: Consumer Acknowledgements and Publisher Confirms
- RabbitMQ: Dead Letter Exchanges
- Microsoft: Competing Consumers pattern
Content reviewed 2026-09-10. Read the editorial policy.