Oblivious HTTP

Bellows is an Oblivious HTTP gateway (RFC 9458) for its own client API. A client encrypts each API call to the attested HPKE key and sends it through a relay. The relay sees the client's address and ciphertext. Bellows sees the request and the relay's address. When a relay is run by someone other than the operator, who is asking is hidden from the operator, and what is asked is hidden from the relay.

Without a relay, a client can still send encapsulated requests straight to Bellows. The content is then unreadable to anything in front of the sealed machine (a load balancer, a TLS terminator, the host), but the operator's network sees the client's address.

A relay and a Bellows run by the same party hide nothing from that party.

The gateway

RouteWhat it does
GET /ohttp-keysThe key configuration as application/ohttp-keys (RFC 9458 section 3.2). It is the HPKE key that /.well-known/bellows attests.
POST /ohttpTakes message/ohttp-req (RFC 9458) or message/ohttp-chunked-req (chunked OHTTP, draft 08) and answers with message/ohttp-res or message/ohttp-chunked-res.

A client must take the key from the attested statement (verifyBellows), not from /ohttp-keys. Anyone in front of the sealed machine can change an unattested key. /ohttp-keys is there for standard OHTTP tooling, and a client that reads it should check that it matches the attested key.

The key suite is DHKEM(X25519, HKDF-SHA256), HKDF-SHA256 and AES-128-GCM. The key pair is made at boot, lives only in memory and is new on every restart. A client that gets a refusal after a restart verifies again and uses the new key.

What an encapsulated request may do

The inner request runs through the same app as a direct request, so it is priced, paid, rate limited and gated by proof of work exactly the same way. It may name these paths, with GET or POST:

Anything else (the operator API, /ohttp itself) gets an encapsulated 404 with code not_found. Another method gets 405.

The inner request keeps its own headers (Authorization, X-Cashu, Content-Type and the rest). Hop headers (Host, Connection, Content-Length, Transfer-Encoding and similar) are dropped. Nothing from the outer request reaches the app: not its headers, not its address.

Replays

RFC 9458 leaves replay protection to the gateway (section 6.5). Bellows refuses a replayed request before it runs:

Admission

Encapsulated requests carry no identity, so the gateway cannot limit one client apart from another. It bounds new requests for everyone together instead: at most BELLOWS_OHTTP_ADMIT_PER_SECOND a second, with a burst of one second's worth. Past it a request gets a plain 503 with Retry-After (whole seconds), takes no place in the replay cache, and can be sent again unchanged once the bound has room. The Bellows relay passes Retry-After on with a 503 and exposes it to browsers; a third-party relay may drop it, so a client should not count on seeing it. Only a request that would be remembered counts: refused, stale and replayed requests use none of it.

A key is kept for at most twice the skew plus one second (601 s by default). Left unset, the bound is the most the replay cache holds over that time (BELLOWS_OHTTP_REPLAY_CACHE_MAX / (2 * BELLOWS_OHTTP_MAX_SKEW_SECONDS + 2), about 332 a second by default), so the cache cannot fill. A flood then competes with honest clients for each second's admissions, rather than shutting every new request out until old keys expire. A sender of more than the bound can still crowd others out while it keeps sending: the bound limits what a flood costs the gateway, not who gets through. A set value that could let the cache fill is refused at startup. Inside the app, each request still meets the same payment, rate limits and proof of work as a direct one. See Run your own.

Refusals

RFC 9458 asks the gateway to answer a request it cannot open with a plain error. Bellows answers with a status and no body:

StatusWhy
415Not an OHTTP content type
400Not decryptable, tampered, cut short, wrong key, malformed, or a replay
413The request is over BELLOWS_OHTTP_MAX_MESSAGE_BYTES
500The answer could not be sealed (a plain answer over BELLOWS_OHTTP_MAX_MESSAGE_BYTES). The request ran. The log line ohttp_answer_failed carries the reason only
503Past the admission bound (Retry-After says when to send again), or the replay cache is full

Each refusal writes one log line, ohttp_refused, with the reason only.

Streaming

Chunked OHTTP streams answers. Bellows seals each piece of the answer as the app writes it (pieces that are ready together are sealed together, never more than 16 KiB in one chunk), so a streamed chat answer reaches the client as it is written. The final chunk is empty, and the padding of the Binary HTTP message goes at its end.

The cryptography is ohttp-ts (MIT): HPKE, the request decryption and the per-chunk sealing with its nonces. Bellows writes the chunk framing itself (a length before each sealed chunk, 0 before the last), because the library's own answer stream seals 16 KiB at a time and holds each chunk until the next one exists. A short chat answer would then arrive only when it ended.

What a streamed answer shows to the relay and the network: the size and timing of each sealed chunk, which follow the tokens the model writes. A direct HTTPS stream shows the same to the network. The total length is padded (at least 1 KiB, then within about 12%). Plain RFC 9458 answers (chunked: false in the client) arrive whole, padded the same way.

Requests are not streamed: the client and the relay send each encapsulated request as one body, because browsers and mobile runtimes do not all stream request bodies.

Size limits

WhatLimit
An encapsulated request, chunked or plainBELLOWS_OHTTP_MAX_MESSAGE_BYTES of Binary HTTP. The gateway reads at most that plus framing room and answers 413 past it. Anyone can send one, so this is the gateway's memory bound
A plain RFC 9458 answerBELLOWS_OHTTP_MAX_MESSAGE_BYTES. It is buffered whole on both sides. The client's plain-mode default is the same 8 MiB, so it opens every plain answer a gateway with the default sends
A chunked answer1 GiB, the chunked OHTTP default of ohttp-ts and of the client. The answer is Bellows' own, so the request limit does not apply to it, and a long stream is not cut short

A plain answer the client cannot open has already run and been charged. Its request id and change URL are inside the answer. Keep the client's maxMessageBytes at least the gateway's BELLOWS_OHTTP_MAX_MESSAGE_BYTES.

The client

import { bellowsFetch, verifyBellows } from './client/index.ts';

const bellows = await verifyBellows('https://bellows.example', { approvedBuilds, verifier });
const operator = { ...bellows, relay: 'https://relay.example/relay/bellows.example' };

const res = await bellowsFetch(operator, '/v1/chat/completions', {
  method: 'POST',
  headers: { 'content-type': 'application/json', authorization: `Bearer ${key}` },
  body: JSON.stringify({ model, messages, stream: true }),
});
// res is the Bellows' answer: status, headers and a body that streams.

bellowsFetch(operator, path, init, options):

The client uses Web APIs only (fetch, WebCrypto, streams) and runs in browsers and in Node. Its only dependencies are ohttp-ts and hpke, which use no Node-only module either; a test checks both.

The relay

relay/ is a small standalone relay: relay/README.md.

Configuration

VariableDefaultMeaning
BELLOWS_RELAYSnoneRelay URLs that forward to this Bellows (each the full URL a client posts to), published in /.well-known/bellows
BELLOWS_OHTTP_MAX_SKEW_SECONDS300How far a request's date may be from the gateway's clock
BELLOWS_OHTTP_REPLAY_CACHE_MAX200000Most encapsulated keys remembered at once
BELLOWS_OHTTP_ADMIT_PER_SECONDthe most the cache holds (about 332)New encapsulated requests admitted per second, for all clients together. See Admission
BELLOWS_OHTTP_MAX_MESSAGE_BYTES8388608Largest Binary HTTP request, and largest plain (not chunked) answer, padding included. Chunked answers are bounded at 1 GiB. See Size limits

Tests

Not verified locally

This page is docs/OHTTP.md in the repository.