API & Backend
RabbitMQ publisher confirms, acknowledgements, and redelivery
A RabbitMQ publisher confirm reports the broker’s acceptance of a publication. A consumer acknowledgement reports that a delivery has been handled. With manual acknowledgements, closing the consuming connection before acknowledging leaves that delivery eligible for redelivery, even when publication was confirmed successfully.
This RabbitMQ 4.1.4 and Pika 1.3.2 example follows an order through publication, disconnect, redelivery, and acknowledgement. It also sends a mandatory publication to a missing queue and requires an unroutable-message error. Together, those checks show what the publisher can know about routing and what the consumer must do after a retry. The single-node lab makes the protocol behavior visible; a production worker also needs a durable business operation and a duplicate-handling policy.
The RabbitMQ example and test runner are available with the recorded results.
Run the RabbitMQ acknowledgement lab
Before you start:
- Install Python 3.11 or newer, Git, and Docker. Confirm the Docker engine is running with
docker info. - Run the commands from the repository root after cloning. The examples use pinned container versions and download them on the first run.
- Use a local development machine with room for disposable containers. The harness removes the resources it creates.
The harness starts a disposable broker, checks readiness as the rabbitmq service user, and runs a Python client in the broker container’s network namespace. It does not publish the broker or management port to your host network.
git clone https://github.com/emminextechdocs/developer-examples.git
cd developer-examples
python3 deep-dives/walkthrough.py rabbitmqThe client image installs the pinned Pika dependency and executes deep-dives/rabbitmq/recovery.py. Containers are removed after the run, and deep-dives/evidence/rabbitmq.json retains the observations. The lab needs network access for images and for installing Pika in each disposable client container.
walkthrough.py runs the lab and prints selected commands and their responses alongside the assertions. Open the full-resolution capture.
Inspect the result before changing the example
The runner writes a fresh JSON record and exits after cleanup. To print only its assertions, run:
python3 - <<'PYTHON'
import json
from pathlib import Path
report = json.loads(Path("deep-dives/evidence/rabbitmq.json").read_text())
assert report["passed"] is True
for event in report["events"]:
if "assertion" in event:
print(event["assertion"], "=>", event["actual"])
PYTHONThis prints each assertion and its observed value from your latest run. If a check fails, fix its setup or implementation and rerun the lab. The following sections explain what the checks establish.
What confirms and acknowledgements mean
The publisher asks the broker to accept a message. The exchange routes it to queues according to bindings and routing rules. A consumer receives a queue delivery. The consumer’s acknowledgement tells the broker that this delivery no longer needs to remain outstanding.
Publisher confirms cover the publishing side. Consumer acknowledgements cover the consuming side. RabbitMQ explicitly describes them as separate mechanisms in its confirmation and acknowledgement guide. An application needs a policy for both if it wants to recover reliably across both connections.
Name the metrics after the operation they measure. A counter called messages_succeeded hides too much. Prefer separate publication, routing-failure, delivery, and business-completion observations. During an incident, these metrics show whether work stopped at publication, delivery, or processing.
Declare a durable queue and persistent message
The client declares a durable orders queue and publishes a message with delivery mode two. Queue durability and message persistence are different settings; neither should be replaced by the vague statement that the client “uses a durable queue.” The queue guide describes queue properties and message states.
This is a single-node fixture. It does not establish replicated availability, disk-loss survival, or recovery from losing an entire broker host. In particular, choosing a durable queue is not the same as choosing a replicated queue type.
For a production design that requires replicated queues, evaluate quorum queues and test the chosen cluster’s failure behavior. Keep that architecture decision separate from this protocol experiment. You can learn acknowledgement semantics from one broker without claiming one broker demonstrates your availability requirements.
Detect an unroutable mandatory publication
The client enables publisher confirms, then publishes through the default exchange with a routing key for a queue that does not exist. It sets mandatory=True. In this configuration, Pika raises UnroutableError, and the test fails if that error is not observed.
The mandatory flag matters because an accepted publication can otherwise be unroutable. A publishing API returning without an exception is not a universal proof that the intended queue received the message. RabbitMQ’s reliability guide discusses routing assurance as a separate concern.
Use a deliberately nonexistent destination in a disposable environment when testing this branch. Do not delete a shared queue to force the error. Preserve the distinction in your application error model: an unroutable message usually needs configuration or destination correction, while a connection failure may leave acceptance uncertain.
Inspect the Pika recovery client
The runnable client contains both failure and success paths. The publisher confirm is obtained before the first consumer read. The first consumer then closes its connection without acknowledging. A second connection waits for the redelivery, acknowledges it, and verifies the queue has no immediately available message.
The complete client is deep-dives/rabbitmq/recovery.py. The runner executes it in the broker’s network namespace so localhost reaches that disposable broker.
"""Publisher confirms, unroutable publications, and manual acknowledgement."""
import pika
parameters = pika.ConnectionParameters("localhost", heartbeat=30, blocked_connection_timeout=10)
connection = pika.BlockingConnection(parameters)
channel = connection.channel()
channel.queue_declare(queue="orders", durable=True)
channel.confirm_delivery()
try:
channel.basic_publish(exchange="", routing_key="missing-queue", body=b"ord-0", mandatory=True)
except pika.exceptions.UnroutableError:
print("PASS unroutable mandatory publication rejected", flush=True)
else:
raise AssertionError("Expected UnroutableError")
channel.basic_publish(exchange="", routing_key="orders", body=b"ord-42", mandatory=True,
properties=pika.BasicProperties(delivery_mode=2, message_id="ord-42"))
print("PASS publisher confirm received; consumer has not acknowledged", flush=True)
method, properties, body = channel.basic_get("orders", auto_ack=False)
assert body == b"ord-42"
assert not method.redelivered
print("PASS first delivery received without acknowledgement", flush=True)
connection.close()
connection = pika.BlockingConnection(parameters)
channel = connection.channel()
import time
deadline = time.monotonic() + 10
while True:
method, properties, body = channel.basic_get("orders", auto_ack=False)
if method is not None:
break
if time.monotonic() > deadline:
raise AssertionError("Unacknowledged message was not requeued")
time.sleep(0.1)
assert body == b"ord-42" and method.redelivered
print("PASS disconnected consumer's message redelivered", flush=True)
channel.basic_ack(method.delivery_tag)
method, _, _ = channel.basic_get("orders", auto_ack=False)
assert method is None
print("PASS acknowledgement removed message from queue", flush=True)
connection.close()The important calls cross different boundaries:
confirm_delivery()enables publisher confirms for this channel.mandatory=Truemakes an unroutable publication visible to the confirmed publishing client.basic_get(..., auto_ack=False)leaves the received message unacknowledged.- Closing the first connection lets RabbitMQ requeue that delivery.
basic_ack(method.delivery_tag)acknowledges the replacement delivery on its own channel.
The client prints these results:
PASS unroutable mandatory publication rejected
PASS publisher confirm received; consumer has not acknowledged
PASS first delivery received without acknowledgement
PASS disconnected consumer's message redelivered
PASS acknowledgement removed message from queueThe fourth line requires both the expected payload and redelivered=True; seeing a second connection succeed is not enough.
Pika’s BlockingConnection reference documents the channel methods and exceptions used here. Keeping the version explicit matters: the article describes the client actually exercised by the fixture, not an unspecified future release.
The test polls with a deadline when waiting for requeueing. Closing a connection initiates broker-side work; one immediately empty basic_get should not be treated as proof that the message disappeared. The deadline keeps the experiment bounded while allowing the broker to complete that transition.
Confirm publication before reading the message
After the mandatory routing test, the publisher sends ord-42 to orders and receives a confirm. At that point the consumer has not acknowledged anything. The next assertion reads the message with auto_ack=False and intentionally leaves the delivery outstanding.
The fixture uses basic_get to control each delivery in the test. Its one-at-a-time sequence does not measure production throughput. A consuming application will usually have a longer-lived delivery loop and a separate concurrency policy.
If your publisher reports success to an upstream caller, define what that response means. It might mean the broker confirmed acceptance, or it might require a later business result. Those contracts have different latency and recovery implications. A publisher confirm alone does not establish that inventory changed, an invoice was issued, or an email was delivered.
Close the connection and verify redelivery
The first connection closes after receiving the message. With manual acknowledgement enabled, RabbitMQ requeues outstanding deliveries when the consuming channel or connection closes. The second connection receives the same payload with its redelivery indicator set.
The test closes the connection deliberately. The broker learns about the close directly. Detecting a dead connection after silent network failure involves additional timing and connection-health behavior, which must be tested separately in your environment.
The redelivery flag is useful evidence about a delivery attempt, but it is not a business deduplication key. The fixture also supplies message_id=ord-42; the application chooses that identifier. RabbitMQ does not automatically enforce your business uniqueness rule merely because the property exists.
Acknowledge after the durable write
The second consumer acknowledges using the delivery tag from its own channel. The next basic_get returns no message. That verifies acknowledgement in this one-message queue. It does not establish anything about an external database because this fixture intentionally has no database mutation.
Delivery tags are scoped to the channel. Acknowledging an unknown tag closes the channel with a protocol error. Always acknowledge on the channel that supplied the delivery. Keep the channel and tag together instead of passing bare numeric tags between worker connections.
For a real worker, choose the acknowledgement boundary explicitly. If a database transaction performs the business change, acknowledge after that transaction commits. If the handler only stores an inbox record for later processing, ensure the inbox is durable and independently recoverable before acknowledging. Our webhook recovery walkthrough examines that second pattern.
Watch the delivery return to Ready in RabbitMQ
The management UI companion holds a real delivery open long enough to inspect. It requires Docker Compose v2. From the repository root:
docker compose -p emminex-rabbit-ui -f deep-dives/rabbitmq/ui/compose.yml up -d- Open
http://127.0.0.1:15674. Sign in withlabandlocal-lab-only. - Select Queues and Streams, then orders-ui. Wait for a statistics refresh.
- Check Ready = 0, Unacked = 1, and Total = 1. The helper published one synthetic order, fetched it, and kept the connection open.
This is a polling basic_get client, so the UI’s registered Consumers count is zero. That does not mean there is no open connection holding a delivery. Stop that client:
docker compose -p emminex-rabbit-ui -f deep-dives/rabbitmq/ui/compose.yml stop consumerAfter the connection closes and statistics refresh, expect Ready = 1 and Unacked = 0. The message is available again; no replacement worker has processed it in this companion.
Remove the companion after inspection:
docker compose -p emminex-rabbit-ui -f deep-dives/rabbitmq/ui/compose.yml down -vHandle a database commit followed by a lost acknowledgement
A database can commit successfully while the following acknowledgement is lost. RabbitMQ then has an outstanding delivery, and recovery can deliver the message again. Reapplying the business operation without a uniqueness rule can duplicate the effect even though the worker’s original transaction succeeded.
Acknowledging before the database transaction merely reverses the risk: the broker can forget the delivery before the business effect exists. Neither ordering makes RabbitMQ and an unrelated database participate in one transaction.
Use a stable business identity and an atomic destination-side check appropriate to the operation. For example, a database transaction can insert an event identity under a unique constraint and perform the associated mutation together. The idempotent webhook article explores that boundary. The recommendation applies to the application; it is not an additional acknowledgement feature.
Set prefetch for push consumers
A real consumer may receive multiple deliveries before acknowledging earlier ones. Prefetch can bound that outstanding population, but the correct value depends on processing time, concurrency, memory, and how much work you want requeued after a failure. A larger prefetch can leave more outstanding deliveries to requeue after failure.
RabbitMQ’s consumer-prefetch guide explains its consumer-scoped behavior. The basic_get fixture does not demonstrate push-consumer prefetch, so do not read its one-at-a-time execution as evidence about a configured QoS limit.
When adding a worker pool, test the maximum in-flight count and the disconnect case together. Record how many deliveries return to the queue when a worker stops. Several consumers disconnecting together can create a sudden backlog of redelivered work.
Limit retries and define a dead-letter policy
Not every failure should immediately return to the same queue. Invalid input may fail forever, while a temporary dependency outage may recover after a delay. Requeueing every rejected message immediately can keep handing the same failing work to consumers without making progress.
Define a retry budget, an error classification, and an operator-visible terminal path. If you use dead-letter exchanges, verify the queue policy and destination behavior. RabbitMQ’s dead-lettering documentation describes when dead-lettering occurs and the safety considerations of republishing.
This lab does not configure a dead-letter exchange, so its successful redelivery is not evidence that an exhausted message will be retained elsewhere. Add a separate test that reaches the retry limit and verifies the terminal destination. Test the terminal destination being unavailable too; a routing diagram alone does not prove the message survives that failure.
Monitor processing alongside connection health
The example configures heartbeats and a blocked-connection timeout. Those settings help bound particular connection behaviors; they do not prove that business processing is progressing. A worker can remain connected while its downstream dependency is stalled.
RabbitMQ’s heartbeat guide explains negotiation and failure detection. Use those semantics when selecting a value rather than treating the number as a universal deadline for every publish or consumer callback.
Instrument the duration and outcome of publication, processing, and acknowledgement separately. During an incident, inspect ready messages, unacknowledged messages, consumer availability, and downstream latency together. An empty ready count can coexist with unfinished work already delivered to consumers. Check the unacknowledged count before concluding that the queue has no remaining work.
Batch acknowledgements only for completed deliveries
The fixture acknowledges one delivery tag at a time. If a production consumer uses the acknowledgement method’s multiple option, it can acknowledge a range of outstanding deliveries on the channel. That is efficient only when the application knows the entire intended range has reached its completion boundary.
A concurrent worker pool can finish deliveries out of order. Acknowledging through a later tag while earlier work is unfinished can tell the broker to stop retaining work that the application still needs to recover. Track completed work before choosing a batched acknowledgement boundary, or use individual acknowledgements where that is simpler to defend.
Test this with two deliberately different processing times. Complete the later delivery first, close the connection, and inspect which messages return. The result should match the application’s documented acknowledgement policy. Do not let a benchmark that counts only completed callbacks conceal what happens to unfinished work after disconnect.
Also distinguish acknowledgement failures from business failures in the logs. A successful business transaction followed by an uncertain acknowledgement needs duplicate-safe recovery. An unsuccessful transaction followed by no acknowledgement needs retry classification. Both can cause redelivery; the destination must check whether the earlier business transaction committed. Log the transaction outcome separately from the acknowledgement outcome so recovery can use that distinction.
Frequently asked questions
Does a RabbitMQ publisher confirm mean a consumer processed the message?
No. Publisher confirms cover the publisher’s interaction with the broker; consumer acknowledgements cover deliveries to consumers. The example receives its publisher confirm before it reads the queue. If an API caller needs the business result rather than broker acceptance, provide a separate way to observe completion and define what happens when that result is delayed.
What happens to unacknowledged messages when a connection closes?
RabbitMQ requeues outstanding manual-acknowledgement deliveries from the closed channel or connection. Another consumer can receive them again. The lab verifies the same order payload and a redelivery flag after a controlled disconnect. A silent network failure also requires the broker to detect the unavailable connection, so its timing can differ from this explicit close.
Does mandatory=True replace publisher confirms?
No. Mandatory publishing exposes a routing failure, while confirms report broker acceptance or rejection. An unroutable mandatory message is returned before its confirm. The Pika example enables confirm mode and expects UnroutableError for the missing queue. Preserve separate error handling for unroutable messages, negative acknowledgements, and connection failures with uncertain acceptance.
Can a consumer acknowledge a delivery on another channel?
No. Delivery tags are scoped to their channel. An acknowledgement on the wrong channel can refer to an unknown tag and close that channel with a protocol error. Keep the delivery’s channel and tag together. When using batched acknowledgements, also confirm that every covered delivery has finished its required durable work.