Serving components

StateSync-GKR proves and verifies. A service around it also admits requests, frames replies, bounds its memory and keeps every core busy between requests. The ssgkr-serving crate in the repository's benches/serving folder packages those parts as a library and composes them into a reference proof server, together with a load frontend and real-TCP checks. It calls only the public statesync-gkr API, so its proofs are byte for byte the proofs of the library's own batch calls. The crate's README (opens in a new tab) is the source of this page.

The reference server is the server behind the v1.2.0 measurements in Serving capacity. This page explains its parts, its modes and the settings those measurements used.

Before you begin#

• Read Production integration. It describes what a service adds around the engine and what it must record.
• Work with prepared circuits, as described in Prepared execution. Every completion helper takes a PreparedSync.
• Production use that generates proofs, embeds or redistributes the proving functionality, or offers proof generation to third parties requires a commercial license; see StateSync-GKR licensing. The crate is distributed under the same Business Source License 1.1 as the engine.

What the crate provides#

Each module works on its own, and the reference server in src/bin/ssgkr-serving composes all of them. Proof bytes do not depend on the mode, thread or order. complete calls make_job_prepared and prove_batch_prepared with a single job, the same per-job path as the library's sequential and parallel batch calls, and every transcript starts from its own job's public inputs. The server repeats this check at startup: each control sample must produce the same bytes through fused completion and through the batch API before the server listens.

The parts it covers#

A service built on the library alone had to write each of these parts itself. With the crate, most of them are components:

Your service still supplies authenticated roots, the state store and the order of updates (Trust boundaries), client authentication and transport security, process supervision and deployment. The crate records stage times and the reference server writes JSON-lines records; exporting metrics to your monitoring system is your code. ContinuousRunner runs its own threads, so an async service hands requests to it through a channel instead of proving on the executor.

Add the crate#

The crate is a standalone Cargo workspace inside the repository. It is not published to crates.io, and nothing in it changes the product workspace, the proof format, the verifier or the transcript order. Build and test it from a checkout:

cargo build --release --locked --manifest-path benches/serving/Cargo.toml
cargo test --release --locked --manifest-path benches/serving/Cargo.toml

To call the components from your own crate, depend on the folder in a checkout pinned to the commit you reviewed. The crate reaches the engine through a relative path in the same checkout, so depend on the engine through that checkout too; two copies of statesync-gkr from different sources would give two incompatible PreparedSync types.

[dependencies]
statesync-gkr = { path = "../statesync-gkr" }
ssgkr-serving = { path = "../statesync-gkr/benches/serving" }

Keep your project outside the checkout, as Installation explains. The crate's lock file uses the dependency versions of the engine's lock file.

A minimal continuous loop, as an excerpt (connection handling, logging and error replies omitted):

use ssgkr_serving::completion::{complete, ContinuousRunner};
use std::sync::{mpsc, Arc};

let (ingress, queue) = mpsc::sync_channel(8192);
let shared = Arc::new((prover, prepared));
let runner = ContinuousRunner::spawn(workers, queue, move |request| {
    let (prover, prepared) = &*shared;
    match complete(prover, prepared, request) {
        Ok(done) => { /* acquire a return permit, then hand done.bytes to the writer */ }
        Err(e) => { /* reply with an explicit error */ }
    }
})?;
// Readers call admission.try_admit() and ingress.try_send(request); dropping
// every sender lets the workers drain the queue and stop.

complete returns CompletionError and never a partial proof. A request of another operation kind than the preparation fails there, because make_job_prepared returns SyncError::KindMismatch in every build profile.

Choose a serving mode#

The reference server has four scheduling modes. Every mode shares the same admission, rejection, framing, return budget and shutdown behavior, and every mode produces the same proof bytes.

The last column comes from three 60-second runs per mode of the same v1.2.0 binary on one 192-core VM, offered 1,000 requests per second from a separate 16-core frontend; v1.1.0, measured the same day with its own server, had p99 433.8–475.1 ms. Serving capacity has the conditions.

• Use continuous when requests are independent and the client matches replies by job identifier. It had the lowest tail at every rate where both v1.2.0 modes met one second. Past the one-connection limit on 192 cores, batch-fused kept completing more requests per second, so size connections as described below.
• Use batch-fused when a client expects replies in batch order.
• Use queued-return or original only to compare against the earlier loop inside one binary.

Run the reference server#

cargo build --release --locked --manifest-path benches/serving/Cargo.toml
target=benches/serving/target/release/ssgkr-serving
"$target" server --out results/server-1 --listen 127.0.0.1:9400 --serving-mode continuous --workers 16 --max-in-flight 14
"$target" load --out results/load-1 --targets 127.0.0.1:9400 --rate 100 --duration-sec 30 --verify-workers 8

Build for the machine that runs the prover, as Installation explains; on the measured host the default release build proved 1.66–1.71 times slower than a native build. Every result directory must be new, because the server and the frontend refuse to overwrite files. The binary also has controls, which runs the startup controls on their own, and sweep-plan.

Size admission, the return budget and workers#

• Start from one worker per core and the default number of jobs in flight. On 48 cores the measured limit was server CPU, so a smaller machine needs its own measurement.
• Plan the reply traffic. Every reply carries one complete encoded proof of about 177 to 187 KB at depth 24, so 3,300 replies per second is about 4.8 Gbit/s. One connection stopped near 4.96 Gbit/s in the measurements, and in 15-second screens two connections to the same server carried 7.30 Gbit/s. Open more connections, or spread clients across servers, when one connection would carry more than that.
• Size the return budget for the replies you let wait. It counts encoded buffer capacity only: jobs in progress, prepared circuits, request frames, thread stacks, socket buffers and allocator overhead are outside it, so measure whole-process memory as well. With the defaults, peak server memory at 5,100 requests per second on 192 cores was 535.2 MiB sampled from /proc (393.5 MiB by GNU time).
• The admission cap decides when overload turns into a refusal. A job admitted beyond what the workers finish within your latency budget will be late, so a lower cap makes overload visible as refusals sooner. The measured runs kept the default of 8,192.

Rejections and overload#

An overloaded server answers with a refusal that the client can count, instead of queueing without limit. A rejected reply carries an empty proof and one of these reasons in its error field:

When admission closes at the run limit or because the stop file appeared, admitted jobs that have not started receive server_run_limit_before_compute, and jobs already computing finish and are sent. terminal.json records which of the two closed the run.

Count every refusal as a failure in your success fractions, together with late and unanswered requests. In the overload runs on 192 cores, the frontend's own limit of 8,192 pending requests held back the excess before the server's admission cap was reached, so the server rejected no request, kept its memory at about a fifth of its limit and exited normally; Serving capacity has the numbers.

Records and stage times#

The server writes metadata.json, server.jsonl and terminal.json; the frontend writes metadata.json, frontend.jsonl, fixtures.jsonl and terminal.json. Reply rows add mode, connection and writer_tid to the fields of the earlier capacity caller, and the fused modes add job_stages:

Use return_enqueue_ns to see replies waiting for the writer or the return budget, and prove_ns to watch the proving step alone. In continuous there are no batch rows; a reply's batch fields describe its own job. Reply headers have the same fields in every mode, so other load callers that speak this frame format accept them unchanged. Durations come from one clock on the worker; never subtract timestamps taken on different hosts.

Check a build#

cargo fmt --manifest-path benches/serving/Cargo.toml --check
cargo clippy --manifest-path benches/serving/Cargo.toml --all-targets --locked -- -D warnings
cargo test --manifest-path benches/serving/Cargo.toml --release --locked
python3 benches/serving/service-checks.py --binary benches/serving/target/release/ssgkr-serving --suite smoke --out results/checks-1

service-checks.py (opens in a new tab) runs real-TCP suites against the built server:

These checks run on one machine and confirm behavior; their latencies are not capacity evidence. The controlled measurements are in Serving capacity.

Next steps#

• Serving capacity: what the reference server sustained on 48, 96 and 192 cores.
• Performance tuning: choose workers, modes, connections and build settings.
• Production integration: failures, records and state changes around the engine.