Recover from a failed webhook delivery

The request succeeds before delivery fails

The API accepts an event and queues a delivery. The receiving application's event store is temporarily unavailable, so it cannot safely accept the webhook and returns 503 Service Unavailable.

Create the delivery and save its ID

curl --request POST \
  https://api.relayline.example/v1/deliveries \
  --header "Authorization: Bearer $RELAYLINE_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "destination": "https://example.com/webhooks/orders",
    "event": "order.created",
    "payload": { "order_id": "ord_7M2L9", "total": 4800, "currency": "USD" },
    "idempotency_key": "order-7M2L9-created"
  }'

{
  "id": "dlv_01J6Y3G8V9K2M4P7Q1R5",
  "event": "order.created",
  "status": "queued",
  "destination": "https://example.com/webhooks/orders",
  "attempts": 0,
  "created_at": "2026-09-04T14:32:18Z",
  "completed_at": null
}

Save dlv_01J6Y3G8V9K2M4P7Q1R5. The request succeeded. In this response, queued means the delivery is waiting for its first attempt. The API reference defines the create and retrieve operations.

Inspect the destination failure

Relayline sends this event to https://example.com/webhooks/orders, with Relayline-Attempt: 1, the delivery ID, and a signature.

{
  "id": "evt_01J6Y5T4M8N2K7P9Q3R1",
  "type": "order.created",
  "delivery_id": "dlv_01J6Y3G8V9K2M4P7Q1R5",
  "created_at": "2026-09-04T14:32:18Z",
  "data": {
    "order_id": "ord_7M2L9",
    "total": 4800,
    "currency": "USD"
  }
}

The receiver cannot store the event and returns:

HTTP/1.1 503 Service Unavailable
Content-Length: 0

This response comes from the webhook destination. It does not undo the successful create request or mean that the caller should create another delivery. Use the delivery ID to find the failed attempt in the receiver's logs and restore its event store.

curl \
  https://api.relayline.example/v1/deliveries/dlv_01J6Y3G8V9K2M4P7Q1R5 \
  --header "Authorization: Bearer $RELAYLINE_API_KEY"

{
  "id": "dlv_01J6Y3G8V9K2M4P7Q1R5",
  "event": "order.created",
  "status": "retrying",
  "destination": "https://example.com/webhooks/orders",
  "attempts": 1,
  "created_at": "2026-09-04T14:32:18Z",
  "completed_at": null
}

A successful GET means the status lookup worked. The body still reports a delivery failure awaiting retry: retrying, with one attempt and no completion timestamp.

Restore the receiver and accept the retry

Relayline makes the second attempt after approximately one minute. No retry request from the caller is needed. The event and delivery IDs stay the same; Relayline-Attempt becomes 2. The signing timestamp and signature are generated again for this attempt. The event's created_at stays unchanged.

Once the receiver's event store is available, handle the retried webhook in this order:

1. Verify the signature against the raw request body and check the timestamp. See the signature verification example.
2. Parse the verified body and validate its event schema before storing it.
3. Insert the event into a durable inbox with a unique constraint on its event ID, evt_01J6Y5T4M8N2K7P9Q3R1. Use an atomic insert that handles conflicts; a separate lookup followed by an insert can race.
4. Have the worker read pending events from that inbox. If you use a separate queue, commit an outbox record in the same database transaction and publish it through a retrying dispatcher. Do not record the event as accepted and then rely on a separate, unprotected queue write.
5. If the event already has a committed inbox record, acknowledge it without inserting a second job. Keep any pending processing attached to the existing record.
6. Return 204 No Content within the 10-second timeout. Do not acknowledge a new event if storage failed.

HTTP/1.1 204 No Content

The create request's idempotency_key prevents duplicate delivery creation for the same project, key, and request data within 24 hours. The inbox prevents repeated webhook attempts from creating duplicate inbox records. Workers can still retry after a crash, so make their database changes and external side effects idempotent too; storing an event ID alone does not guarantee exactly-once processing.

Confirm the second attempt succeeded

Retrieve the same delivery again after the retry:

curl \
  https://api.relayline.example/v1/deliveries/dlv_01J6Y3G8V9K2M4P7Q1R5 \
  --header "Authorization: Bearer $RELAYLINE_API_KEY"

{
  "id": "dlv_01J6Y3G8V9K2M4P7Q1R5",
  "event": "order.created",
  "status": "delivered",
  "destination": "https://example.com/webhooks/orders",
  "attempts": 2,
  "created_at": "2026-09-04T14:32:18Z",
  "completed_at": "2026-09-04T14:33:19Z"
}

In this example, Relayline receives the 204 before the timeout. The delivery ID is unchanged, attempts is 2, and completed_at is set. delivered confirms the destination acknowledged the webhook; it does not confirm that its asynchronous order processing has finished.

If the state is still queued or retrying, wait before checking again. If a status request returns 429, honor Retry-After as described in Errors and rate limits.

If the destination does not recover

Relayline makes at most five attempts. After the first retry, the remaining delays are approximately 5 minutes, 30 minutes, and 2 hours. If all attempts fail, retrieving the delivery returns a terminal result like this:

{
  "id": "dlv_01J6Y3G8V9K2M4P7Q1R5",
  "event": "order.created",
  "status": "failed",
  "destination": "https://example.com/webhooks/orders",
  "attempts": 5,
  "created_at": "2026-09-04T14:32:18Z",
  "completed_at": "2026-09-04T17:08:19Z"
}

Automatic retries have ended. Stop waiting for this delivery to change, inspect the receiver's logs using the delivery ID, and fix the underlying problem. A failed result means Relayline did not receive a successful acknowledgement within the allowed attempts. The receiver may still have stored or processed the event if its response was lost. Check its records before deciding how to recover the business operation.

The sample API has no manual replay endpoint. Repeating the original create request with the same idempotency key and body within its 24-hour retention window returns the original delivery; it does not restart a failed one.

See the full retry schedule and idempotency rules. Errors from the create API itself follow the separate client retry guidance.