Delivery & Reliability

Overview

When a domain event matches an active webhook subscription, SynthesQ creates a delivery record and begins the delivery lifecycle. Each delivery is an independent unit of work with its own status, retry history, and unique identifier. The platform handles retries automatically using exponential backoff and disables subscriptions that fail consistently to protect both your endpoint and the delivery pipeline.

Delivery Lifecycle

Every delivery passes through a defined set of statuses:

Status	Description	Transitions
pending	Queued for delivery; waiting for an available worker	-> `delivering`
delivering	HTTP request in progress to the subscriber endpoint	-> `delivered`, -> `failed`
delivered	Endpoint returned a 2xx response; delivery complete	Terminal
failed	All retry attempts exhausted without a successful delivery	Terminal

A delivery is considered successful when your endpoint returns any HTTP status code in the 2xx range. Any other response code (including 3xx redirects) is treated as a failure and triggers a retry.

Retry Strategy

Failed deliveries are retried according to an exponential backoff schedule. The delay between attempts increases with each failure to give transient issues time to resolve without overwhelming your endpoint.

Backoff Schedule

Attempt	Delay After Failure	Cumulative Time
1 (initial)	Immediate	0 minutes
2	1 minute	1 minute
3	5 minutes	6 minutes
4	30 minutes	36 minutes
5	120 minutes	156 minutes (~2.5 hours)

After the fifth attempt fails, the delivery transitions to failed status and no further retries are made.

Retry Timing

Delays are approximate. Retries are processed by background workers and may experience slight variance depending on queue depth. The actual delay is always at least the scheduled interval but may be slightly longer.

Auto-Disable Behaviour

To protect both the delivery pipeline and your infrastructure, SynthesQ automatically disables subscriptions that exhibit sustained failure. The auto-disable policy evaluates the following criteria:

Criterion	Threshold
Failure rate	Greater than 95% of deliveries failed
Time window	Rolling 24-hour period
Minimum deliveries	At least 10 delivery attempts in the window

When all three criteria are met, the subscription transitions to disabled status. No further deliveries are queued until you manually re-activate the subscription via POST /api/v1/webhooks/subscriptions/{id}/activate.

Investigate Before Re-Activating

Auto-disable indicates a systemic issue with your endpoint - not a transient glitch. Before re-activating, verify that your endpoint is reachable, returning 2xx responses, and processing requests within the timeout window. Re-activating without fixing the root cause will result in another disable cycle.

Events that occur while a subscription is disabled are not retroactively delivered when the subscription is re-activated. Use the Event Feed to catch up on missed events after resolving the issue.

Manual Retry

You can manually retry a specific failed delivery. This creates a new delivery attempt regardless of whether the original delivery has exhausted its automatic retries.

Endpoint: POST /api/v1/webhooks/subscriptions/{subscriptionId}/deliveries/{deliveryId}/retry

Authentication: Required (Bearer token)

Response (200 OK):

json

{
  "success": true,
  "message": "Delivery retry queued successfully",
  "data": {
    "delivery_id": "01jdlvr789abc012def345ghi6",
    "status": "pending",
    "attempt": 6,
    "updated_at": "2026-03-20T14:00:00Z"
  },
  "meta": {}
}

Debugging with Manual Retry

Manual retries are useful during development and incident recovery. After fixing your endpoint, retry a known failed delivery to confirm the fix before re-activating the subscription.

Idempotency

Every delivery includes an X-Webhook-Delivery-Id header containing a unique identifier for that specific delivery attempt. Use this ID to implement idempotent processing in your consumer.

Headers sent with every delivery:

Header	Description
`Content-Type`	`application/json`
`X-Webhook-Signature`	HMAC-SHA256 hex digest of the request body
`X-Webhook-Delivery-Id`	Unique ULID for this delivery attempt
`User-Agent`	`SynthesQ-Webhooks/1.0`

Because retries and manual retries can cause the same event to be delivered multiple times, your consumer should check whether it has already processed a given X-Webhook-Delivery-Id before taking action:

javascript

app.post('/webhooks/synthesq', async (req, res) => {
  const deliveryId = req.headers['x-webhook-delivery-id'];

  // Check if already processed
  if (await isDeliveryProcessed(deliveryId)) {
    return res.status(200).json({ received: true, duplicate: true });
  }

  // Process the event
  await processEvent(req.body);

  // Mark as processed
  await markDeliveryProcessed(deliveryId);

  res.status(200).json({ received: true });
});

Always Return 200 for Duplicates

Even if you detect a duplicate delivery, return a 2xx response. Returning an error code causes the platform to retry again, creating an unnecessary retry loop.

Delivery Inspection

List Deliveries

Returns delivery attempts for a subscription, ordered by most recent first.

Endpoint: GET /api/v1/webhooks/subscriptions/{subscriptionId}/deliveries

Authentication: Required (Bearer token)

Query Parameters:

Parameter	Type	Description
`status`	string	Filter by status: `pending`, `delivering`, `delivered`, `failed`
`event_type`	string	Filter by event type (exact match)
`per_page`	integer	Results per page (default: 25, max: 100)
`page`	integer	Page number

Response (200 OK):

json

{
  "success": true,
  "data": [
    {
      "id": "01jdlvr789abc012def345ghi6",
      "subscription_id": "01jwebhook123abc456def789",
      "event_type": "product.created",
      "entity_id": "01jprod789abc012def345ghi6",
      "status": "delivered",
      "attempts": 1,
      "last_attempt_at": "2026-03-15T10:30:01Z",
      "last_response_code": 200,
      "last_response_time_ms": 142,
      "created_at": "2026-03-15T10:30:00Z"
    },
    {
      "id": "01jdlvr790abc013def346ghi7",
      "subscription_id": "01jwebhook123abc456def789",
      "event_type": "product.price_changed",
      "entity_id": "01jprod789abc012def345ghi6",
      "status": "failed",
      "attempts": 5,
      "last_attempt_at": "2026-03-15T13:06:00Z",
      "last_response_code": 503,
      "last_response_time_ms": 30012,
      "created_at": "2026-03-15T11:00:00Z"
    }
  ],
  "meta": {
    "current_page": 1,
    "per_page": 25,
    "total": 2,
    "last_page": 1
  },
  "links": {
    "first": "/api/v1/webhooks/subscriptions/01jwebhook123abc456def789/deliveries?page=1",
    "last": "/api/v1/webhooks/subscriptions/01jwebhook123abc456def789/deliveries?page=1",
    "prev": null,
    "next": null
  }
}

Get Delivery Details

Returns the full delivery record including the event payload and attempt history.

Endpoint: GET /api/v1/webhooks/subscriptions/{subscriptionId}/deliveries/{deliveryId}

Authentication: Required (Bearer token)

Response (200 OK):

json

{
  "success": true,
  "data": {
    "id": "01jdlvr790abc013def346ghi7",
    "subscription_id": "01jwebhook123abc456def789",
    "event_type": "product.price_changed",
    "entity_type": "product",
    "entity_id": "01jprod789abc012def345ghi6",
    "status": "failed",
    "payload": {
      "id": "01jprod789abc012def345ghi6",
      "name": "Wireless Keyboard",
      "sku": "KB-WIRELESS-001",
      "selling_price": 44.99,
      "currency": "USD"
    },
    "attempts": [
      {
        "attempt": 1,
        "attempted_at": "2026-03-15T11:00:00Z",
        "response_code": 503,
        "response_time_ms": 30012,
        "error": "Service Unavailable"
      },
      {
        "attempt": 2,
        "attempted_at": "2026-03-15T11:01:00Z",
        "response_code": 503,
        "response_time_ms": 30008,
        "error": "Service Unavailable"
      },
      {
        "attempt": 3,
        "attempted_at": "2026-03-15T11:06:00Z",
        "response_code": 503,
        "response_time_ms": 30015,
        "error": "Service Unavailable"
      },
      {
        "attempt": 4,
        "attempted_at": "2026-03-15T11:36:00Z",
        "response_code": 503,
        "response_time_ms": 30010,
        "error": "Service Unavailable"
      },
      {
        "attempt": 5,
        "attempted_at": "2026-03-15T13:36:00Z",
        "response_code": 503,
        "response_time_ms": 30012,
        "error": "Service Unavailable"
      }
    ],
    "created_at": "2026-03-15T11:00:00Z",
    "updated_at": "2026-03-15T13:36:00Z"
  },
  "meta": {}
}

Troubleshooting

Deliveries Stuck in Pending

Symptom: Deliveries remain in pending status for more than a few seconds.

Possible Causes:

The background worker queue is under heavy load
The queue worker process has stopped

Resolution:

Check delivery status: GET /api/v1/webhooks/subscriptions/{subscriptionId}/deliveries/{deliveryId}
If deliveries remain pending for more than five minutes, contact your system administrator to verify queue worker health

High Failure Rate Leading to Auto-Disable

Symptom: Subscription transitions to disabled without manual intervention.

Possible Causes:

Your endpoint is down or unreachable
Your endpoint is returning non-2xx responses (e.g., 401, 500, 503)
Network issues between SynthesQ and your endpoint
Your endpoint is timing out under load

Resolution:

Review recent deliveries: GET /api/v1/webhooks/subscriptions/{id}/deliveries?status=failed
Check last_response_code and attempt details for error patterns
Fix the endpoint issue
Test with a manual retry: POST /api/v1/webhooks/subscriptions/{subscriptionId}/deliveries/{deliveryId}/retry
Once confirmed working, re-activate: POST /api/v1/webhooks/subscriptions/{id}/activate
Use the Event Feed to catch up on events missed during the disabled period

Duplicate Events Received

Symptom: Your consumer processes the same event more than once.

Cause: Retries or manual retries can deliver the same event multiple times. This is expected behaviour in an at-least-once delivery system.

Resolution: Implement idempotent processing using the X-Webhook-Delivery-Id header as described in the Idempotency section above.

Managing Subscriptions - Create subscriptions, configure event patterns, and verify signatures
Event Feed - Poll for events using cursor-based pagination
Webhooks Overview - Module architecture and quick start

Delivery & Reliability ​

Overview ​

Delivery Lifecycle ​

Retry Strategy ​

Backoff Schedule ​

Auto-Disable Behaviour ​

Manual Retry ​

Idempotency ​

Delivery Inspection ​

List Deliveries ​

Get Delivery Details ​

Troubleshooting ​

Deliveries Stuck in Pending ​

High Failure Rate Leading to Auto-Disable ​

Duplicate Events Received ​

Related Documentation ​

Delivery & Reliability

Overview

Delivery Lifecycle

Retry Strategy

Backoff Schedule

Auto-Disable Behaviour

Manual Retry

Idempotency

Delivery Inspection

List Deliveries

Get Delivery Details

Troubleshooting

Deliveries Stuck in Pending

High Failure Rate Leading to Auto-Disable

Duplicate Events Received

Related Documentation