Scaling & high availability

MCP REST Bridge runs happily as a single process — one Bun instance with a local SQLite file handles a lot. When you need redundancy or more throughput, it scales horizontally: run several identical instances behind a load balancer, coordinated through a shared SQLite database.

The model

• Stateless request handling. Each tool call is self-contained; any instance can serve any REST call.
• SQLite is the coordination layer. Admin config, guards, keys, audit, usage and the HA primitives all live in the database. Point every instance at the sameDB_PATH (shared storage / a shared volume) so they see one config.
• Opt-in HA flags turn on cross-instance behaviour (below) — they're off by default so a single node stays simple.

Identical instances behind a load balancer, coordinated through one shared SQLite. Each still proxies to your REST & MCP backends; background loops run on the elected leader only.

Turn on the HA primitives

Setting	Effect
RATE_LIMIT_SHARED=true	Rate limits use SQLite fixed-window counters, so a per-tool limit is enforced across all instances, not per-process.
REGISTRY_SYNC=true	Each instance periodically reconciles its live registry from SQLite — a client registered (or removed) on one node propagates to the others.

Background loops that must run once — alert evaluation, maintenance schedules, and the health-check/auto-eviction loop — elect a single leader automatically via a SQLite lease. This isn't a flag; it's always on and needs no configuration.

Load balancing your backends

Separately from scaling the bridge itself, a single client can fan out across several backend targets (N-way load balancing), configured per client from the admin API. A target that fails is skipped for LB_TARGET_COOLDOWN_MS (default 30s) before it's tried again. Combine with per-client canary/failover (see Guardrails & resilience) for graceful degradation.

MCP sessions & sticky routing

The Streamable HTTP transport keeps per-session state in memory on the instance that opened the session. Two options:

• Sticky sessions — enable session affinity on your load balancer for the MCP endpoints (/mcp, /mcp/:name, /mcp-custom/:bundle) so a session stays on one instance. Recommended for streaming clients.
• Stateless calls — clients that open a fresh request per call don't need affinity and balance freely.

REST proxying and the admin API need no affinity.

Caveats to know

• Shared SQLite requires shared storage. SQLite over a network filesystem has locking quirks; prefer a volume all instances mount locally, or keep writes modest. For very high write volume, run fewer, larger instances.

• Every guard except the rate limiter counts per instance. RATE_LIMIT_SHARED=true moves rate counters into SQLite; nothing else moves. Circuit breakers, the response cache, in-flight request coalescing and load-balancer target cooldowns all live in the memory of the process that saw the traffic. With N instances that means, concretely:

  Guard	Effect at N instances
  Circuit breaker	A failing backend must fail failureThreshold times per instance before all of them open
  Response cache	Up to N copies of the same entry; hit rate drops roughly N-fold for evenly-balanced traffic
  Request coalescing	Collapses concurrent duplicates within one instance, not across them
  LB target cooldown	A target marked down on one instance is still tried by the others until they fail too

  None of these is a correctness bug — each instance converges on the same decision, just independently and after its own evidence. Size CIRCUIT_BREAKER_FAILURE_THRESHOLD and cache TTLs with the replica count in mind, and prefer sticky sessions (above) if cache hit rate matters.

• The audit hash-chain is per-instance. Its tamper-evidence (verifyAuditChain) assumes one writer; cross-instance chain integrity is out of scope — stream to a SIEM (AUDIT_SINK_URL) for a consolidated, ordered record instead.

• A health-eviction hold is per-instance too, if you route with /livez instead of /readyz. Under the default (leader-only, /readyz-gated) topology this doesn't matter — only the leader ever probes backends or serves traffic. But if you switch readiness to /livez so every replica serves (see the Helm chart's readiness-probe override →), a client re-registration can land on a non-leader replica and only clear that replica's in-memory eviction mark, leaving the leader's own mark (and therefore its reconcileFromDb()) stuck withholding the client. Re-register against the leader, or forgetClient/re-register through a session pinned to it, to clear the hold.

Checklist

• [ ] All instances share one DB_PATH
• [ ] RATE_LIMIT_SHARED=true and REGISTRY_SYNC=true
• [ ] Load balancer health-checks /health (or /livez) for routing REST/MCP data-plane traffic — request handling is stateless, so every instance is fit to serve regardless of leadership. /readyz is 200 only on the current leader (leader lease + DB); pointing a throughput-scaling LB at it takes every other instance out of rotation. Reserve /readyz-gated routing for a deliberate active/passive failover topology, and use /livez for the orchestrator's liveness/restart probe
• [ ] Sticky sessions for the MCP endpoints /mcp · /mcp/:name · /mcp-custom/:bundle (if you use streaming)
• [ ] AUDIT_SINK_URL set for a consolidated audit trail

See Deployment → for the container setup and Configuration → for every flag.