Skip to content
Long TailLong Tail

Errors become escalations

The escalation boundaryDashboardHTTPSDKMCPconditionaldidInterruptproxyActivities

When retries run out, the failure becomes a row an operator can act on. The run waits for retry or skip instead of disappearing.

The situation

A payment capture fails after its retry budget. The order should neither be dropped nor retried forever. Someone in operations should see the error, decide, and the workflow should continue from that decision.

The recipe

workflows/capture-order.ts
import { Durable } from '@hotmeshio/hotmesh';
import { conditional, type LTEnvelope } from '@hotmeshio/long-tail';
import * as activities from './activities';


const { capturePayment } = Durable.workflow.proxyActivities<typeof activities>({
  activities,
  retry: { maximumAttempts: 3, backoffCoefficient: 2, maximumInterval: '30s' },
});


export async function captureOrder(envelope: LTEnvelope) {
  const { orderId } = envelope.data;
  const { workflowId } = Durable.workflow.workflowInfo();
  for (let attempt = 1; ; attempt++) {
    try {
      return { type: 'return' as const, data: await capturePayment(orderId) };
    } catch (err) {
      if (Durable.workflow.didInterrupt(err as Error)) throw err;
      const next = await conditional<{ action: 'retry' | 'skip' }>(`capture-error-${workflowId}-${attempt}`, {
        role: 'ops',
        subtype: 'error',
        description: `Capture failed: ${(err as Error).message}`,
        metadata: { orderId, attempt },
        envelope: { error: (err as Error).message },
      });
      if (!next || next.action === 'skip') return { type: 'return' as const, data: { orderId, skipped: true } };
    }
  }
}

didInterrupt passes the engine’s own control-flow signals through. Every real failure parks one ops row with the message, and each attempt gets its own deterministic signal id.

The guarantee

A failure is a pending row with its error attached, searchable by orderId and subtype. See Workflows.

The escalation boundary

Ask for what is missing

When a run lacks a fact it needs, it asks the one person who has it with a form for just that field, then continues.

conditionalassigneeDashboardHTTPMCP

The escalation boundary

Answer, timeout or cancel

Every wait ends in one of three typed values: the answer, false on timeout, or null on cancel. Each branch is explicit in code.

conditionalDashboardHTTPSDK

The escalation boundary

A checklist computed at runtime

The workflow decides which steps a person must confirm. The form renders them from the row, so one schema serves every variation.

conditionalx-lt-widgetDashboardSDK

The physical world

Offline, quiet and alarm rows

A stopping alarm, a dropped machine and a silent one each become a row for the line operator, resolved when the machine recovers.

line-alarmmachine-offlineMQTTDashboard