The theory behind Mark 1, published as working papers from Mark 1 by Omium Research: what a verified outcome costs, how to
verify without a test suite, and how agents can share what they learn without sharing code.
Paper I
Tokens per Verified Outcome
An economic theory and benchmark protocol for agentic coding systems.
What you pay for a coding agent is not the price of a request. It is every token in the session, divided
by the outcomes that were actually verified. The paper defines that metric, derives when checking an
agent's work is worth what the check costs, and shows why today's leaderboards can't report it.
A break-even for verification
A verifier lowers cost only if its discrimination clears a threshold set by how often the agent fails,
what an attempt costs, and how expensive a wrong answer is to find later.
v* = (1 − p)·D·(α − β) − c·(1 − α)
How many repair rounds
One formula gives the cost-optimal number of self-repair rounds. With nothing fitted it returns 2 at
function scale and 6 at repository scale, matching the two knees reported in the literature.
Not recoverable from pass@k
Resolution rate and mean cost per instance cannot be turned into tokens per verified outcome. Two
systems can report identical numbers and cost very different amounts.
Waste, in four channels
Session waste splits exactly into iteration, exploration, misallocation and context, so you can see
where the tokens went, not only how many.
Cost of one verified outcome, tokens
No verifier 300.0k
LLM judge · AUROC 0.537 360.0k +20%
LLM judge · AUROC 0.65 257.1k −14%
Grounded rubric · AUC 0.886 214.7k −28%
Model-implied at p = 0.4, c = 30k, D = 150k tokens. The weaker judge is worse than having no verifier at
all, even if it were free. Paper I, Table 1.
Paper II
Verification Without Oracles
Calibrated outcome adjudication for software agents in test-poor environments.
Agents misreport their own success: across roughly ten thousand measured trajectories, 35.6% of task
failures on τ²-bench were cases where the agent declared success, and no LLM judge caught them above AUROC
0.65. Published verifiers are tested on repositories that have test suites. Most production repositories
don't. This paper combines weak signals into one calibrated verdict, with no labeled data.
Recoverable from agreement alone
With three or more independent groups of checks, the class prior and every check's error rates can be
estimated from unlabeled agreement. A check that can only reject correctly, like a compile failure,
fixes the orientation.
Four verdicts beat two
A checker that may answer satisfied, violated, refused or error has strictly lower expected cost than
any forced yes-or-no rule whenever some cases fall in the uncertain band. Reporting a checker outage as "your agent failed" carries a provable excess
risk.
Self-certifying labels
The same estimates give the noise rate of the verdicts themselves, which gates whether they are fit to
train other models on.
Abstaining pays
In the worked example, the four-verdict rule costs about 192.6k tokens per verified outcome, against
245.9k for a strict binary rule and 208.2k for a lenient one, and 36% below having no verifier.
One ensemble, every verdict
Rubric agent
Reward model
Anchor
Posterior
Verdict
accept
accept
accept
0.934
satisfied
accept
reject
accept
0.612
refused
reject
accept
accept
0.385
violated
reject
reject
accept
0.065
violated
any
any
reject
0
violated
checker outage: no signal
error
86.1% decided13.9%
Worked example: a rubric agent, a reasoning reward model and a compile-and-type anchor, prior 0.4. Exactly
one pattern lands in the abstention band, and that is the refused share. Paper II, Section 12.
Paper III
Federated Experience
Privacy-preserving, cross-organization procedural memory for coding agents.
Lessons from one company's agent sessions can make another company's agent better. The raw sessions can't
move: model vendors are contractually barred from training on customer content, and mixing unfiltered
self-generated successes into training has been shown to hurt. The paper shares distilled procedure
instead, never trajectories or code.
Artifacts, not trajectories
Workflow templates, pitfall and remedy pairs, invariants and routing hints. A written grammar and
static checks keep code, identifiers and free text out. Only those artifacts, with capped statistics, cross the tenant
boundary.
Privacy you can cite
The shared pool has no tenant identifier, so every released view is (k, m)-anonymous by construction.
Statistics are released with differential-privacy noise, and each tenant's budget is accounted for.
Do no harm
An artifact is admitted only when its measured benefit clears a lower confidence bound. With high
probability every admitted artifact has positive utility, so on average the pool does no worse than
private memory.
Travels only as far as it should
Retrieval discounts an artifact by the distance between the source and target task distributions. In
the worked example, a new tenant gets a guaranteed 3.8% expected saving on day one.
Why curation is not optional
+6.6 pts
resolve rate for +7.5% tokens, about 9% fewer tokens per resolved issue, when distilled experience is
injected and same-repository experience is excluded.
15.3% → 8.7%
what one strong open agent scored after training on unfiltered self-generated successes.
1Stays localSessions, trajectories and verified labels never leave your side.
2AbstractLiterals, paths, identifiers and free text become typed placeholders.
3AdmitKept only if its measured benefit clears a lower confidence bound above zero.
4ReleaseOnly when k artifacts from m tenants share its coarse profile, with noise on the counts.
5RetrieveUsed by another tenant only as far as its tasks resemble the source.
Reported figures are cited from the literature in Paper III, Section 1. The pipeline is Sections 4, 5 and 8.
Working papers, v1, August 2026. Anurag Upadhyay, Mark 1 by Omium Research. Papers I to III of the Mark I Protocol series.
These are theory, algorithms and evaluation protocols. Every figure above is a closed-form computation from a
paper's stated model, calibrated to values published in the literature it cites. None is a measurement of Mark 1,
and the papers don't describe what the current app does.