Webhook Retry Strategies

Strategy: Fixed Interval

The simplest approach — retry every N seconds. Wait 60 seconds, try again. Wait 60 seconds, try again.

// Fixed interval — simple but problematic
const RETRY_INTERVAL = 60_000; // 60 seconds
const MAX_RETRIES = 5;

async function deliverWithFixedRetry(url, payload, attempt = 1) {
  try {
    const res = await fetch(url, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(payload),
    });
    if (!res.ok) throw new Error('HTTP ' + res.status);
  } catch (err) {
    if (attempt >= MAX_RETRIES) throw err;
    await sleep(RETRY_INTERVAL);
    return deliverWithFixedRetry(url, payload, attempt + 1);
  }
}

Problem: thundering herd. If 10,000 webhooks fail at the same time (e.g. a destination goes down), they all retry at exactly the same moment — 60 seconds later. Then again at 120 seconds. The destination gets hammered with synchronized bursts, making recovery harder.

Strategy: Exponential Backoff

Double the delay between each retry. This gives the destination progressively more time to recover.

Much better than fixed intervals — early retries catch transient blips quickly, while later retries back off for longer outages. But it still has the thundering herd problem if many webhooks fail simultaneously.

Strategy: Exponential Backoff + Jitter (The Right Answer)

Add a random offset (jitter) to each delay. This spreads retries across time, eliminating synchronized bursts.

// Exponential backoff with jitter — the production-ready approach
async function deliverWithBackoff(url, payload, attempt = 1, maxRetries = 5) {
  try {
    const res = await fetch(url, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(payload),
    });
    if (res.status === 429) {
      // Respect Retry-After header if present
      const retryAfter = res.headers.get('Retry-After');
      if (retryAfter) {
        await sleep(parseInt(retryAfter, 10) * 1000);
        return deliverWithBackoff(url, payload, attempt, maxRetries);
      }
    }
    if (!res.ok) throw new Error('HTTP ' + res.status);
  } catch (err) {
    if (attempt >= maxRetries) throw err;
    const baseDelay = Math.pow(2, attempt) * 1000; // 2s, 4s, 8s, 16s...
    const jitter = baseDelay * 0.2 * Math.random(); // 0-20% random jitter
    await sleep(baseDelay + jitter);
    return deliverWithBackoff(url, payload, attempt + 1, maxRetries);
  }
}

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

This is the pattern used by AWS, Google Cloud, and most production webhook systems. The jitter prevents correlated retries from overwhelming the destination during recovery.

Strategy: Dead Letter Queue

What happens after all retries fail? The event should never be silently dropped. Send it to a dead letter queue (DLQ) for manual review, alerting, or later replay.

async function deliverWebhook(event) {
  try {
    await deliverWithBackoff(event.url, event.payload);
    await markDelivered(event.id);
  } catch (err) {
    // All retries exhausted — move to dead letter queue
    await sendToDeadLetterQueue({
      eventId: event.id,
      url: event.url,
      payload: event.payload,
      lastError: err.message,
      failedAt: new Date().toISOString(),
      attempts: event.maxRetries,
    });
    await alertOpsTeam(event); // Slack notification, PagerDuty, etc.
  }
}

A DLQ gives you a safety net. When the destination comes back online, you can replay failed events from the DLQ — either manually or automatically. No data is lost.