Benchmark Report — AI Agents (v1.10.0)¶
Date: 2026-07-12
Package: mineproductivity.agents (software v1.10.0)
Scenarios: benchmark/scenarios/agents/
Environment: Windows 11 (AMD64), CPython 3.12.10, mineproductivity editable-installed. Single process, no other load control — reference numbers for order-of-magnitude regression tracking, not a controlled lab measurement.
Both scenarios are standalone scripts (the mineproductivity.benchmark harness package is not yet implemented — see benchmark/README.md); re-run them with:
python benchmark/scenarios/agents/task_repository_latency.py
python benchmark/scenarios/agents/workflow_parallel_throughput.py
1. TaskRepository.get()/list() latency¶
Populations span "a shift" (10³ tasks) through "a quarter, never pruned" (10⁵). get() figures are means over 10,000 calls; list() over 5.
| tasks | get (µs) | list all (ms) | list by_scope (ms) | scope matches |
|---|---|---|---|---|
| 1,000 | 0.09 | 0.01 | 0.29 | 500 |
| 10,000 | 0.08 | 0.10 | 2.57 | 5,000 |
| 100,000 | 0.08 | 1.54 | 28.74 | 50,000 |
Reading: get() — the hot per-dispatch path — is flat (~0.08 µs) across a 100× population spread, the O(1) bound design spec §36 requires. list() is linear by core.BaseRepository's own contract — the discovery path (§23), not the dispatch hot path. Identical shape to the Optimization/Simulation repository benchmarks, as expected under the shared core.InMemoryRepository reference.
2. Task-dispatch throughput (sequential vs. parallel)¶
The same N independent tasks dispatched through TaskExecutor.execute (the per-task path WorkflowEngine.run loops over): policy gate → act → persist → audit. The example-local agent is a trivial fast echo, isolating executor overhead from reasoning-backend cost. Parallel runs over an 8-worker ThreadPoolExecutor.
| tasks | sequential (ms) | seq tasks/s | parallel (ms) | par tasks/s |
|---|---|---|---|---|
| 50 | 1.27 | 39,237 | 5.13 | 9,747 |
| 200 | 5.50 | 36,385 | 10.16 | 19,688 |
| 1,000 | 24.55 | 40,734 | 56.55 | 17,682 |
Reading: sequential throughput is ~36k–41k dispatches/s and scales linearly — the executor's per-task overhead is flat. The threaded path is slower here, and honestly so: with a sub-microsecond pure-Python agent, every dispatch holds the GIL, and the shared AgentAuditTrail.record() append is a deliberate serialization point (§21). The benchmark's point is correctness under concurrency, not speedup: independent tasks (distinct ids) complete without contention, lost updates, or audit corruption (design spec §32). Wall-clock speedup materializes only when the agent's reasoning backend releases the GIL (network/LLM I/O, native compute) — exactly the reasoning-backend boundary this package keeps out of its own code (§3.1, §4).
Regression guidance¶
Re-run both scenarios before each release touching agents; investigate if get() exceeds ~1 µs at any population, or if sequential dispatch throughput drops below ~10k tasks/s for the trivial agent on comparable hardware (a sign the gate/dispatch/persist/audit path regressed).