​

Egress reliability

Status: specification. This describes the target contract and the layers that enforce it. Where a layer is not yet implemented, that is remaining work, not a description of current behaviour. See ../roadmap/milestones.md (repository) for sequencing.

​

The failure taxonomy

​

Class 1 — transport / infra failures

​

Class 2 — semantic / quality failures

​

The egress contract boundary

Persistence order is a design decision this spec does not silently assume. The current write path persists the outbound event after delivery (see ../persistence.md for why). A typed egress boundary is orthogonal to that ordering — it can enforce “no EgressError reaches a user” whether the durable row is written before or after the wire send. If a future revision moves persistence before delivery (to get retry/idempotency off a durable record), that is target behaviour and must define failure-state, retry, idempotency, and duplicate-handling semantics before it is adopted — persist-then-send and send-then-persist have materially different partial-delivery semantics. This spec does not depend on that change; it only requires the typed boundary to sit on the one send path that already exists.

An egress payload is either a deliverable agent message or a structured error event. A structured error event is never rendered to a user channel. It is logged, attributed, retried, or escalated — but a connector’s user-facing send path only ever accepts the first variant.

// Illustrative shape; the point is that these are distinct types and the
// user-facing delivery path only accepts the first.
type Egress interface{ isEgress() }

type AgentMessage struct {
    Body        string       // canonical Markdown, pre-render
    Attribution Attribution  // which RuntimeAgent perspective, which conversation
    // …
}

type EgressError struct {
    Class      FailureClass // Transport | Semantic
    Retryable  bool
    Cause      error        // full internal detail, for logs/errtrack only
    Attempt    int
    // …
}

func (AgentMessage) isEgress() {}
func (EgressError) isEgress()  {}

// Deliver is the typed PRE-RENDER boundary. It accepts only AgentMessage; an
// EgressError does not typecheck here and is routed to logs / error-tracking /
// the retry loop instead. Deliver does not bypass the existing connector
// contract: it renders canonical Markdown to Rendered parts and sends each part
// through connector.Outbound.Send, preserving the connector's chunking and
// size-limit handling exactly as SendReplyCommitting does today. The type gate
// is added *around* the existing Render+Send path, not in place of it.
func (c Connector) Deliver(ctx context.Context, msg AgentMessage) (Receipt, error) {
    rendered := c.Render(msg.Body)          // canonical Markdown -> Rendered parts
    // ... cross commit boundary, then Send each rendered part in order,
    //     identical to the current SendReplyCommitting semantics.
}

​

Layer A — classify, retry, fall back at the model-call layer

1. Classify the failure as retryable or terminal.
   • Retryable: 5xx, 429 (honour Retry-After), connection reset, request timeout, empty completion, transient JSON-parse on a tool call. (A text-protocol tool decision that fails to parse first gets one in-attempt repair from the connector, which re-asks with the model’s reply and a correction; only if that also fails does the transient error reach this loop. See docs/connectors/model.md.)
   • Terminal: 400 with a real validation error, auth failure, a hard context overflow that a retry cannot fix without changing the request.
2. Retry retryable failures with bounded attempts and exponential backoff plus jitter. Bounded on three axes, not one:
   • a maximum attempt count;
   • a maximum per-wait delay — a provider-supplied Retry-After is honoured but clamped to this ceiling. Retry-After is provider-controlled; an un-clamped large value lets a provider pin an interactive turn and a worker slot open for as long as it likes, which is a denial-of-service the contract hands to the upstream. Clamp it;
   • a total recovery deadline for the turn. When the deadline is reached, recovery stops and proceeds to Layer C regardless of remaining attempts. An unbounded retry loop — in attempts or in wall-clock — is a new outage, not a fix.
3. Fall back across routes when retries on the primary are exhausted, or as soon as a primary attempt times out (a stalled route): another OpenRouter route, another provider, or a smaller/cheaper model for a degraded-but-real answer. Fallback order is policy, not hardcoded.
4. Only when retries and fallbacks are exhausted does the failure become a structured EgressError and enter Layer C.

​

Shipped model deadline policy

• With a later route configured, the route is abandoned at its first attempt timeout and the next fallback starts immediately; there is no second attempt on the stalled route. This was chosen over “one more try” because production stalls come in bursts: without it, a stalled primary spent attempt after attempt at one attempt timeout each (~3 minutes at the defaults) and the chain was never reached.
• With no later route (no fallback configured, or the last fallback), a timed-out attempt is retried on the same route like any retryable failure, bounded by the attempt count, route share, and total deadline. There is nothing else to try, and failing earlier would not give the user a better outcome.
• Non-timeout retryable failures (429, 5xx, transport resets) always retry the same route with backoff first; they are usually momentary and the primary is the preferred model.

​

Layer B — the egress contract boundary

​

Layer C — graceful degradation policy

• Async / proactive contexts (a scheduled digest, an unprompted heads-up, a background job posting a result): silence beats a fallback. A human who is heads-down and hits a wall does not send “something went wrong” — they just don’t send. A failed proactive message is logged and retried on the next cycle, not announced. This composes with Mesh’s existing observe/engage model: not emitting is already a first-class, explainable outcome.
• Interactive contexts (a user is waiting on a direct reply): a warm holding line — “give me a sec on this, hitting a snag” — or an explicit handoff, never the raw internal string. The holding line is a real product decision with a follow-up commitment attached (see commitments.md (repository)), not a reworded error.
• Never, in any context, the internal cause. The full detail lives in logs and error-tracking (see ../error-tracking.md), attributed to the perspective and run, where an operator can act on it.

​

The egress quality gate

• Heuristic-gated, not blanket. Cheap deterministic checks run first — regex and simple classifiers for known leak shapes (a “As an AI language model” refusal opener, a leaked <thinking> block, an apology-only reply to a substantive ask). Only a candidate flagged by a heuristic is escalated to the fast-model judge. The judge never runs on all traffic.
• Bounded and non-fatal, but fail-safe on high-confidence leaks. The judge is a model call and can itself fail. Its failure mode is defined and depends on the confidence of the heuristic that flagged the candidate:
  • Low-confidence candidates (tone, a possible-but-ambiguous refusal): a judge timeout or error does not block the send — it degrades to “send the candidate, flag it for offline review.” The cost of a false hold outweighs the cost of shipping a marginal message.
  • High-confidence deterministic matches (a leaked <thinking> block, a verbatim system-prompt fragment): these are routed to Layer C or a safe sanitization path, never sent on judge failure. Sending the candidate because the judge timed out would leak exactly the content the gate exists to block — the fail-open default must not apply to the cases we already know are unsafe. For these, the judge only ever downgrades a block (confirms it is benign); it is never the thing standing between a known leak and the user.
  Either way the judge’s own failure is bounded and never becomes a second user-facing error. Who guards the guard is answered by making the guard’s failure a no-op for the ambiguous case and a fail-closed for the unambiguous one, not a second outage in either.
• Opt-in per install/agent. The gate has a latency and cost profile; it is a policy an operator turns on for surfaces where quality matters more than the last 200–800ms, not a global default.

​

Relationship to the ingress classifier

​

What this is not

• It is not an LLM reading every outgoing message. That is the expensive tool for the cheap problem; Layer A handles the common case deterministically.
• It is not a global gate that adds latency to healthy replies. Layers A and B cost nothing on the happy path; the Class 2 gate is opt-in and heuristic-gated.
• It is not a promise that Mesh always has an answer. It is a promise that when Mesh does not, the user sees a human-shaped silence or a warm holding line — never the machine’s own error text.

​

Design summary