Messages move independently of workers

• Producer

  Software that creates messages for another part of a system to process.

  Example: A shop adds an order-confirmation task to a queue.

• Work queue

  A place where tasks wait until a worker can process them.

  Example: Confirmation emails wait in a queue during a busy sale.

• Worker

  Software that takes a queued task and performs the work.

  Example: An email worker sends an order confirmation.

• Dead-letter queue

  A separate destination for messages that could not be processed under the configured retry policy.

  Example: A repeatedly failing task is parked for investigation instead of retried forever.

• Worker B

  A second worker processing tasks from the system.

  Example: Another worker handles a delivered task; duplicate-safe processing can prevent repeating an effect.

Delivery + acknowledgement

Processing completion and broker acknowledgement are distinct steps.

1. Producer Work queue: Job 42. A job is accepted under the broker’s publish policy.
2. Work queue Worker: Deliver 42. Delivery transfers work to a consumer.
3. Worker Work queue: ACK 42. After durable processing, the worker acknowledges.

Redelivery + deduplication

At-least-once delivery can repeat a message; idempotent processing prevents repeated effects.

1. Producer Work queue: Job 42. The broker accepts the job under its configured publish policy.
2. Work queue Worker: Deliver 42. The worker commits the effect but loses its acknowledgement.
3. Work queue Worker B: Redeliver 42. After detecting failure, the broker redelivers.
4. Worker B Work queue: ACK duplicate. The consumer recognizes the completed idempotency key.

Finite retries dead letter

A configured dead-letter policy isolates a repeatedly failing message.

1. Producer Work queue: Poison job. A malformed job enters the queue.
2. Work queue Worker: Attempt 1. The consumer cannot process the message.
3. Worker Work queue: Retry policy. A bounded application policy permits another attempt.
4. Work queue Worker: Last attempt. The finite retry budget is exhausted.
5. Work queue Dead-letter queue: Dead-letter route. Configured routing sends the message for investigation.

Backpressure

Prefetch limits unacknowledged deliveries; producer throttling needs its own flow-control policy.

1. Producer Work queue: Burst of jobs. Work arrives faster than this worker can finish.
2. Work queue Worker: Prefetch: 1. Limit this consumer to one unacknowledged delivery.
3. Work queue Producer: Flow-control signal. A configured capacity policy slows further publishing.
4. Worker Work queue: ACK. Completion frees capacity for another delivery.

Competing consumers

A work queue distributes deliveries; it does not broadcast each job to every worker.

1. Producer Work queue: Jobs A + B. Two distinct jobs enter the work queue.
2. Work queue Worker: Job A. One available consumer receives A.
3. Work queue Worker B: Job B. Another consumer receives B.
4. Worker B Work queue: ACK B. Completion order can differ from publication order.
5. Worker Work queue: ACK A. The other job completes later.