Reliability

Gateway retries some failed requests on a fresh connection. A retry can reach your target a second time, so design every endpoint with side effects to be safe to repeat.

TimeoutsCopy Anchor Link

Each phase of a request has its own timeout. There is also a total limit, which applies to retries.

The total limit is checked before each retry starts. It is not a hard deadline for an attempt already in progress, so one request can take longer than 120 seconds in total when the connect, send and read phases all run close to their own limits.

A timeout stops Gateway waiting. The target may still have completed the work.

RetriesCopy Anchor Link

Retries on a status code apply only to the methods listed above. Retries after a connection or response failure follow different rules:

• If sending the request to the target fails after part of a POST or other non-idempotent request has been sent, Gateway returns 502 and does not retry.
• If the target accepts the request but then closes the connection without a response, sends a response Gateway cannot read or parse, or the read fails in some cases, Gateway can retry the request on another connection regardless of method. The target may already have acted on the first attempt.
• A retry opens a fresh connection rather than reusing the failed one.
• A response that is too large is not retried, because the result would be the same. See Responses.

Make targets safe to repeatCopy Anchor Link

Treat every effectful endpoint behind Gateway as one that can receive the same request more than once.

• Accept an idempotency key from the client, for example in an Idempotency-Key header, and store the result of the first request made with each key.
• When a request arrives with a key you have already processed, return the stored result instead of repeating the effect.
• Use the same approach for timeouts: a timeout does not prove the target did nothing.

ExampleCopy Anchor Link

The total limit and the per-phase timeouts are set per route. In the current Gateway configuration format they look like this.

{
  "routes": [
    {
      "path_prefix": "/api/",
      "upstream": "api",
      "connect_timeout_ms": 5000,
      "send_timeout_ms": 30000,
      "read_timeout_ms": 60000,
      "total_timeout_ms": 120000
    }
  ]
}