Benchmarks

Before and after AgentTx, measured.

Every chart renders the committed results file. The same scripted agent and the same tools run with and without AgentTx; nothing on this page is typed in by hand.

AMD Ryzen 5 5600H with Radeon Graphics · 12 cores · 14.5 GBrustc 1.98.1release build5aa5567 + uncommitted changesSep 15, 2026, 05:16 PM UTC Raw JSON

cargo run --release -p agenttx-bench -- --out benchmarks/results/latest.json

1 · Agent recovery

What happens after a step fails.

A naive loop retries the failing step, then restarts from scratch with raw errors in context. AgentTx rewinds to the right step with a one-line hint. Pick a scenario and size.

An early step stores the wrong customer id. The mistake only surfaces many steps later as a foreign-key violation. Bad data at step 8; failure at step 24 of 32.

Tool calls

4917% fewer

vs 59 without AgentTx

Feedback tokens

2797% fewer

vs 1,029 without AgentTx

Modeled latency

49 s17% fewer

vs 59 s without AgentTx

Side-effect damage

0none left

vs 5 without AgentTx

Tool calls by workflow size

Lower is better. Each row runs the same agent and tools twice.

Without AgentTx
With AgentTx

Without AgentTx

completed

Tool calls: 59; 32 planned
Failed calls: 5
Feedback: 1,029 tok; 4,113 B
Left wrong: 5; 4 dup · 0 leaked · 1 files

restart ×1retry ×3skip duplicate ×1

1step 24: retry with raw error in context
2step 24: retry with raw error in context
3step 24: retry with raw error in context
4step 24: restart from step 1
5step 1: duplicate on replay, treated as done

With AgentTx

committed

Tool calls: 49; 32 planned
Failed calls: 1
Feedback: 27 tok; 107 B
Left wrong: 0; 0 dup · 0 leaked · 0 files

dependency jump ×1

1step 24: dependency_jump → resume at step 8

2 · Step latency

The price of a safety net, per step.

Microseconds of overhead per tool call, next to an assumed 1 s LLM round-trip that dominates real agent latency.

Direct tool call (before)

3.36 µs

p50 · p99 5.87 µs

AgentTx engine, embedded (after)

52.5 µs

+49.2 µs per step · 0.005% of a 1 s LLM call

AgentTx over gRPC (after)

348 µs

+345 µs per step · 0.03% of a 1 s LLM call

Per-step latency: kv.put

Log scale. What a single tool call costs with and without the transaction layer. The added time buys snapshots, the undo journal, dependency tracking and rollback.

p50 (bar)
p95 tick · p99 whisker

Direct write (no protocol): Before: the tool's effect as one auto-committed RocksDB put. No transaction, snapshot, journal or rollback.
AgentTx engine (in-process): After: snapshot pointer, journaled overlay write, dependency-graph update and output record, embedded as a library.
AgentTx over gRPC (loopback): After, as a proxy: full ExecuteStep round-trip over HTTP/2 on localhost, including protobuf encoding and the engine.

3 · Rewind & commit

Rewinds scale with changes, not database size.

The undo journal replays prior values in one atomic batch. Commit publishes the overlay and validates write-write conflicts.

Snapshot restore vs. the writes it undoes

Rewinding 100K journaled writes takes 291 ms (p50) — 44% of the time the writes took. Log–log scale.

Restore (p50)
Original writes (p50)

Commit latency

Atomic publish with write-write conflict checks and cleanup of journal, snapshots and Saga logs (p50).

4 · Error cleaner

Kilobytes of stack trace in, one line out.

Real traces from Java, Python, Node.js, Go and Rust services, reduced deterministically in microseconds. Select one to compare.

Context size: raw error → Clean Hint

Log scale. Bytes an agent would add to its prompt for each real-world trace, before and after cleaning.

Raw error (before)
Clean Hint (after)

Javapg_foreign_keyforeign key violationkey: user_idparsed in 7.18 µs (p50)

Before — 2.15 KB, 24 lines

org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [orders_user_id_fkey]
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:276)
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:233)
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at com.acme.orders.OrderService$$SpringCGLIB$$0.createOrder(<generated>)
	at com.acme.agent.tools.CreateOrderTool.execute(CreateOrderTool.java:58)
	at com.acme.agent.runtime.ToolDispatcher.dispatch(ToolDispatcher.java:112)
Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:95)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:56)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3375)
	... 42 more
Caused by: org.postgresql.util.PSQLException: ERROR: insert or update on table "orders" violates foreign key constraint "orders_user_id_fkey"
  Detail: Key (user_id)=(101) is not present in table "users".
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498)
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:152)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	... 51 more

5 · Throughput

Parallel transactions, serialized steps.

Each transaction holds its own lock, so independent agents scale across cores.

Throughput: concurrent transactions

In-process engine, 20 steps + commit per transaction. Peak 44K steps/s at concurrency 4.

Methodology

How these numbers were produced.

Harness version 0.1.0. Full details in the benchmark methodology guide.

Agent model: Deterministic scripted agent (no LLM). It fixes the root-cause step once any feedback in its context names the offending field, and follows the step the runner asks for next.
Without AgentTx (baseline): Common ReAct-style loop without a transaction layer: append the raw error to the context, retry the failing step up to 3 times, then restart from step 1 (up to 3 restarts). Each tool call auto-commits and side effects fire immediately. Duplicate-key errors on replay are treated as already done.
With AgentTx: Same agent and tools through the AgentTx engine with the default rollback policy (local depth 2, 1 global reset, replay budget factor 2.0). The agent receives only the Clean Hint.
Tokens: Tokens are estimated as ceil(bytes / 4).
Modeled latency: Modeled end-to-end latency = tool calls × assumed LLM latency per call + measured harness wall time.