Responses
Buffered responses, size limits and what this means for streaming.
Gateway reads the complete response from the target before sending it to the client.
Size limit
Responses, headers included, are limited to 32 MB by default for each route. If a response is larger, Gateway returns 502 Bad Gateway and does not retry.
Streaming
Because responses are buffered, the client receives the response only when the target has finished. Gateway does not currently stream responses, so token-by-token model output, server-sent events and streaming gRPC are not passed through as they are produced.
If your client depends on receiving partial output as it is generated, call that target directly for now.