SovereignClaw Benchmarks

All numbers below were measured live on the 0G Galileo testnet + the free-tier 0G Compute Router against the tagged Phase 8 code. Scripts, methodology and raw JSON are committed so anyone can reproduce or diff them on their own machine.

• Raw JSON: scripts/.benchmarks/*.json
• Scripts: scripts/benchmark-*.ts
• Run script: pnpm benchmark:<name>

Targets referenced below are from the roadmap, §16 "Performance benchmarks". A yes means we met the target; a NO is documented honestly with the measured number + the bounded reason.

Summary table

1. Cold start — scripts/benchmark-cold-start.ts

Times the clone → first-run path the quickstart prescribes:

pnpm install                              (offline; node_modules + pnpm cache)
forge install + forge build               (contracts/lib/* + ABIs)
pnpm --filter core/memory/inft build      (tsup bundles)
examples/research-claw pnpm dev           (live testnet run)

Measured on Linux x86_64, Node v23.3.0, lockfile hot:

research-claw-run dominates the budget; the rest is deterministic and well under a minute. Invocation: pnpm benchmark:cold-start. Pass --clean for a true cold run (wipes node_modules and dist/), --skip-run to skip the live step (no faucet spend), or --with-studio to additionally start the backend and run pnpm smoke:studio (mints 3 iNFTs + manifest; Phase 7 carryover landed in Phase 8).

Known flake. The research-claw-run step occasionally needs a retry because a free-testnet 0G Storage upload reverts with status=0 when an indexer node advertises a stale fee (pinned SDK v1.2.1). See the Phase 4 entry in dev-log.md for the full write-up.

2. Lines of code — scripts/benchmark-loc.ts

Counts non-blank, non-comment lines so the §16 target is measured against API surface, not example scaffolding. The minimal snippets are committed inside the benchmark script itself so this file is the single source of truth.

The hand-written examples are intentionally larger than the minimal snippets — they include dotenv loading, structured JSON logging, event listener wiring, iNFT mint flow, replay checks, and cleanup. Those are valuable for a DoD demo but not part of the API surface claim.

Invocation: pnpm benchmark:loc. Pass --check to exit non-zero if any target is exceeded (suitable for CI).

3. Inference RTT — scripts/benchmark-inference-rtt.ts

Measures the round-trip of a TEE-verified one-shot chat completion against the 0G Compute Router — the same code path sealed0GInference uses. The cold measurement is the first request in a fresh process; the warm median is over subsequent requests with connection pooling kept alive.

With N=3, model=qwen/qwen-2.5-7b-instruct, prompt "What year was the Transformer paper published? One short sentence.":

Note on tee_verified: null. The free-tier router currently returns responses without a tee_verified field in the trace, even when the model is TEE-attested on the backend. This is a known provider limitation, not a SovereignClaw regression; sealed0GInference reports the raw value back to the caller, who can decide whether to accept. See the Phase 0–1 entries in dev-log.md.

Invocation: pnpm benchmark:inference-rtt --n 3. Use --delay-ms 0 on self-hosted routers; the default 2 s delay avoids the free tier's rate-limit (3 req / short window).

4. Revocation latency — scripts/benchmark-revoke-latency.ts

Measures the "click-to-unreadable" latency of revokeMemory(...) against a running backend oracle and 0G Galileo.

t0 = "click" (revokeMemory called)
│
├─ POST /oracle/revoke                   ← oracle refuses future reencrypt
│                                          (few hundred ms; not separately
│                                           timed in this run — see below)
│
├─ AgentNFT.revoke(...)                  ← chain-durable unreadable
│   wait for receipt                       (t2; bounded by Galileo block time)
│
└─ post-revoke /oracle/reencrypt          ← client-observable refusal
    expect OracleRevokedError              (t3 = t0 + observedRefuseMs)

Phase 9 instrumentation (revokeMemory now emits onPhase hooks + returns per-phase timings) lets us report three distinct numbers directly, no guess-work.

With N=1 (one mint + one revoke, freshly minted throwaway iNFT, measured 2026-05-03 on 0G Galileo):

The "<5 s" target is met for the definition of "unreadable" that matches a real user's threat model: the moment the oracle will refuse to re-encrypt the token. That happens inside a single revokeMemory(...) call, roughly one HTTP round-trip after the user clicks. The oracle-refused hook fires at that moment, and the measurement excludes wallet-sign time (~300 ms on this hardware).

Why chain-durable still misses. 0G Galileo produces a block every ~2 s and revokeMemory waits for one full confirmation. A single confirmation of a revoke tx on an unloaded testnet reliably lands in 6–12 s. That is a physical property of the chain, not a SovereignClaw defect; a block-time reduction at the 0G layer would close the gap automatically. For a deployment that absolutely requires chain-durable revocation within 5 s, roll onto a chain with sub-5 s finality.

All three numbers are published so readers can pick the definition that matches their threat model. See docs/security.md §5 for the semantics.

Invocation: pnpm benchmark:revoke-latency --n 1. Requires apps/backend running on http://localhost:8787 and a funded wallet (see .env.example).

5. Mesh throughput — scripts/benchmark-mesh-throughput.ts

Runs planExecuteCritique sequentially N times against the real 0G Compute Router with a 3-agent mesh and reports tasks/second.

With N=3, maxRounds=1, task-delay-ms=30000, accept=0:

Why we miss the 0.5 target. Each task issues 3 sequential inference calls (planner → executor → critic), and the free-tier router's per-request latency is ~1.5–2 s. 3 × ~1.7 s = ~5 s per task ≡ 0.2 tasks/s, which is what we measure.

To hit 0.5 tasks/s the router round-trip would need to be under ~600 ms, or the pattern would need to parallelise planner & executor speculation, or the model would need to be faster. On a higher-throughput router or local TGI instance the effective number climbs sharply (we have not committed numbers for those environments; the script accepts --task-delay-ms 0 and any COMPUTE_ROUTER_BASE_URL so you can rerun).

The mesh coordination overhead (Bus append, eventKey, SeqCounter) is measured separately in packages/mesh/test/ unit tests at the sub-ms level; it is not the bottleneck.

Invocation: pnpm benchmark:mesh-throughput --n 3 --max-rounds 1 --task-delay-ms 30000. Adjust the delay upward if you see HTTP 429s from the router; adjust it to 0 on self-hosted routers.

6. Studio deploy — scripts/smoke-studio-deploy.ts

Measured as part of Phase 7 DoD. Full graph → validated code → 0G Storage manifest + 3 iNFTs minted:

See the Phase 7 entry in dev-log.md for explorer URLs.

Methodology notes

• No warm caches across reported numbers. Each script either sleeps

through the provider's rate-limit window or explicitly measures a cold path. No benchmark re-uses state from a previous run unless it is clearly labeled as "warm".

• No retries. A failed HTTP or on-chain call aborts the benchmark.

Published numbers are always from a clean sweep.

• Linux x86_64, Node v23.3.0. Re-runs on macOS or Node 22 LTS are

welcome; please include the host profile in the JSON summary when contributing numbers.

• Raw JSON committed. scripts/.benchmarks/*.json contains the

full per-sample breakdown, enough to recompute every header number in this file.

If you find a number suspicious, the script that produced it is the authority. Open an issue with your JSON attached and we'll diff.