Serving capacity

This page reports what one server sustains with version 1.2.0 when every request travels the whole path. A request is offered on a fixed schedule, proved by the reference server from Serving components, sent as a complete encoded proof over TCP to a separate frontend and accepted there against its original request. Each version comparison pairs v1.2.0 with v1.1.0 measured on the same host type on the same day. The results were measured on the released version.

Results at a glance#

Capacity within one second means the highest offered rate at which every repeat accepted every timed request within 1 s, judged only on runs where the frontend was not the limit. For v1.1.0 the same-day grid was coarse, so its limit lies between the value shown and the next rate tried; the earlier campaign failed at 650 requests/s on 48 cores and at 1,150 on 192 cores.

Capacity within one second#

192 cores and the network limit#

Each reply carries one complete encoded proof: 176,948, 181,646 and 186,992 bytes for membership, non-membership and update at depth 24. At 3,300 replies per second that is 4.82 Gbit/s from the server to the frontend.

Over one connection, the highest rate with every request within one second was 3,300 requests/s, three 60-second runs in each v1.2.0 mode, with p99 35.4–38.5 ms in continuous mode and 225.1–300.3 ms batch-fused. At 3,400 requests/s the server's send rate levelled at about 4.96 Gbit/s; in every run up to that rate, server CPU stayed at or below 50% and frontend CPU at or below 52% over the run. In 15-second screens with two connections to the same server, every request up to 5,000 per second was accepted within one second at 7.30 Gbit/s. The one-connection limit therefore came from the network path of a single connection between the two VMs, not from proving. A deployment that needs more from one server opens more connections, as the two-connection runs did.

At 5,000 requests/s over two connections, frontend CPU reached 74%, the level at which these studies mark the frontend as a possible limit, so 5,000 is a lower bound for the server. Past the one-connection limit, continuous runs completed about 2,000 to 2,200 requests per second and batch-fused runs about 3,270 to 3,290, both below the offered rate.

48 and 96 cores#

On 48 cores, server CPU reached 96% at the limit, so proving set it. On 96 cores, the server still had CPU to spare at the limit: its return budget held all 192 permitted replies, workers waited 259 seconds in total for a permit, and the reply traffic matched the one-connection limit seen on 192 cores. On the same day, v1.1.0 met one second at 500 requests/s on both machines and missed it at 1,000, with 7.9% (48 cores) and 21.1% (96 cores) of requests within one second.

With the 16-core frontend, the 48-core server met one second at 1,600 requests/s with p99 34.7 ms. At 1,800 and 2,000 it still did, but the frontend ran at 73% and 81% CPU, so those runs were judged on the 48-core frontend instead. The highest completion rates in the table likewise come from runs whose frontend was not the limit.

Two 48-core servers or one 96-core server#

At 500 and 1,000 requests/s with the same total worker resources and the 16-core frontend, v1.2.0 accepted every request within one second in all twelve runs, with p99 between 33.3 and 34.1 ms in both deployments. With v1.1.0, one 96-core worker failed at 1,000 requests/s. With v1.2.0, the choice between one larger server and two smaller ones no longer decides the tail at these rates.

Tail latency at a fixed load#

Every run in this table accepted every timed request within one second. In continuous mode the p99 stays at 33 to 34 ms from 100 to 1,150 requests per second: a request no longer waits for a batch to fill or for the slowest proof of a batch. At 1,150 requests/s the median job took 18.7 ms from dispatch to finish on its worker.

The earlier v1.1.0 campaign met one second for only 70.37% of requests at 1,150 requests/s, while v1.1.0 on the v1.2.0 measurement day met it for all of them. Both are shown in the next figure.

Where the gain comes from#

The serving components can run the v1.1.0 prover code, and the v1.2.0 binary can run the earlier serving loop, so the two changes were also measured apart. At 1,000 requests/s on 192 cores, three 60-second runs each:

Either change lowers the tail by itself, and together they lower it most. One cause in the earlier loop was measured directly. After the last reply of a full batch, the loop paused for a median 45.50 ms before it could start the next batch; with release timers added at 1,150 requests/s, freeing the batch's intermediate data on one thread took 45.27 ms of that pause. In both v1.2.0 modes, the worker that computed a job also frees it, in a median of 22.5 to 23.8 µs per job. At 1,000 requests/s v1.2.0 continuous kept 20.6 of the 192 cores busy on average, against 65.1 for v1.1.0.

Long runs and a second host#

The 30-minute run at 3,300 requests/s accepted all 5,940,000 timed requests within one second, with p50 26.7 ms, p99 35.3 ms and a maximum of 58.8 ms. Straight-line fits over the run show the queue and the tail flat and memory almost flat: pending requests −0.006 per minute, per-second p99 −0.006 ms per minute and server memory +0.08 MiB per minute.

A second 192-core VM repeated the confirmation: three 60-second runs at 3,300 requests/s met one second for every request, with p99 37.6–46.4 ms, and 1,000 requests/s gave p99 34.1 ms. v1.1.0 on that VM had p99 666.8 ms at 1,000 requests/s from the same 48-core frontend.

On 48 cores, the 15-minute run of the earlier campaign was repeated at 500 requests/s:

Overload: bounded memory#

Two runs drove the 192-core server past its capacity on purpose:

Peak server memory is the largest resident memory of the server process sampled from /proc during the run; GNU time reported a lower peak for the same process, shown in brackets. In both runs the server's memory stayed far below the limit, about a fifth of it in the memory-limited run, and both processes exited normally. Finished replies wait under the return budget: when the frontend cannot take replies fast enough, workers wait for a permit instead of growing a queue of replies, as the full budget in the 96-core runs above showed. The frontend reached its own limit of 8,192 pending requests before the server's admission cap filled, so the excess was held back on the frontend and the server rejected nothing; in the send-limited run, 4,113 requests had no reply when the 15-second drain ended. These requests count as failures in the success fractions above.

When the admission cap does fill, the server refuses each further request with server_total_queue_full instead of queueing it; Serving components lists every refusal and the checks that exercise them. In the earlier v1.1.0 campaign at 2,000 requests/s with the 16-core frontend, 14,105 requests met that frontend limit and were never sent; with v1.2.0 none were.

Compute cost per million requests#

The estimate divides the on-demand hourly price of each VM type, checked on 25 October 2026 ($10.35 an hour for the 192-core VM), by the requests served in an hour at the highest rate with every request within one second, and assumes continuous operation at that rate. Storage, network transfer, idle time and tax are excluded, and it is not an invoice. On the same day v1.1.0 was not offered more than 1,150 requests/s, so its cost is an upper bound.

Conditions#

On the v1.2.0 measurement day, v1.1.0 completed proofs within 1.1% of its earlier campaign's compute rate on 192 cores, so the earlier campaign's v1.1.0 series stays comparable on the same axes. Changing the frontend from 16 to 48 cores left v1.2.0 at p99 33.9 ms at 1,000 requests/s and did not lower v1.1.0's tail (458–649 ms), so the tail differences come from the server. Benchmark methodology describes the statistics and outcome classes.

Earlier v1.1.0 serving results#

The serving results of the earlier v1.1.0 campaign, including its 48-core fine-tuning grid, are kept as their own series in Benchmark results. They are never divided by a later run.

Next steps#

• Serving components: the parts and settings of the reference server.
• Performance tuning: choose workers, modes, connections and build settings.
• Benchmark methodology: statistics, timers and hardware.