The Oxbox Survey

A survey of free and low-cost models

What is free or cheap on OpenRouter, ZenMux, Requesty and OpenCode Zen this week, what it costs you in data, and what happened when we pointed some of them to the oxbox source code. Every claim here says how it was learned: measured from a catalog, observed in a run, or reported by somebody else. These are small tests by one person on one codebase; read the numbers as first impressions, not as measurements of the models.

Editor's notes

I had never heard of free models until Ox Alpha came along. I tend to use the most expensive frontier models for anything I care about because I trust them more and feel like I have learned how they operate, but "free" is a very tempting price point. What if I could use them for routine tasks like reviewing code? Since I don't trust them at all, not even a little bit, I built oxbox to sandbox them, and decided to only try them out on code that's already out in public view. I thought it might be interesting to share the results of my experiments, which turned into this survey report.

I'll run this periodically - once a week is my ambition - and give you my take on what I find. The editor's notes are written by me, and the rest is written by Claude Fable with review and editing by me. I hope you find it useful!

--Curtis

Highlights • Top models • What it costs • Full model list • What we ran • How far to trust this • Sources

This week's highlights

• The first issue. This is our first released issue after a few trial runs, so you'll see references to models we found in those previous experiments.
• MiniMax M3 free looks decent.
  • Why you should care: if you want a free code reviewer this week, this is the one with at least a little evidence behind it.
  • minimax/minimax-m3:free reviewed oxbox's own code. 20 of 27 findings reviewed are real, including a stale-lock spin at full CPU, a retry path that read the previous attempt's status file, and a cross-host redirect that could turn a private repository's exposure verdict into "public". On three batches where not every finding was tested, six are confirmed by the fixes oxbox shipped the same day and the rest are unverified, so no rate is claimed for them. Details under "What we ran".
• Two free models were given a byte-identical review payload, and they ranked the same defects in opposite orders.
  • Why you should care: in our one matched test, a model's own severity label was no guide to which findings were real. Read the finding, not the label.
  • MiniMax M3 and OpenCode Zen's nemotron-3-ultra-free both received the two scripts behind oxbox's exposure gate, the check that decides whether a repository is already public before any code is sent to a model, where a wrong "public" would publish private code. Same files, same prompt. Both found the redirect defect. MiniMax put it first and called it the only finding that produces the worst-case outcome; Nemotron filed it eighth as UNCERTAIN and led with a "critical" finding that two lines of the code refute. Seven of MiniMax's twelve findings were real; two of Nemotron's ten were.
• A third of the priced free tier could not be called today, for reasons no catalog carries.
  • Why you should care: before you plan around a free row, call it once; the catalog will not tell you.
  • The survey asked every free model on OpenRouter, ZenMux and Requesty one question. Of 38, 17 answered. Eight OpenRouter rows returned 404: No endpoints available matching your guardrail restrictions and data policy, which names the survey account's privacy settings rather than the model. Both Thinking Machines rows, inkling and inkling-small, answered 403: only available on agentic harnesses, a client gate the catalog does not mention. On Requesty, the provider behind poolside/laguna-m.1 and poolside/laguna-xs.2 said "please check the model you provided", and nvidia/nemotron-3-nano-30b-a3b, which expired on OpenRouter on 2026-08-24, answered 410 Gone. Four more were rate-limited, two returned upstream errors, and the two Lyria audio previews rejected a text request. Every model that did answer found the planted defect, except mistral/leanstral-1-5, which named the line before it, and the Nemotron content-safety classifier, which classified the prompt instead of answering it.
• The free tier is volatile.
  • Why you should care: If you use free models, don't expect them to last long.
  • Of the 42 free rows that the three priced venues listed on 2026-08-24, seven were gone eight days later and three had arrived, leaving 38. In theory, paid models should be less volatile, but we haven't confirmed that yet. Either way, a manifest lets oxbox fall back to paid entries (--failover --allow-paid) when a free model disappears.
• The Stripe acquisition of OpenRouter is agreed but not yet closed.
  • Why you should care: Stripe might not be as interested in giving out free tokens as a scrappy startup.
  • OpenRouter's announcement of 2026-08-19 says the deal is "subject to customary closing conditions" and expected to close "in the coming weeks", promises "same mission, same name, same product, same roadmap", but says nothing about pricing or the free tier. No closing had been reported when this edition was written.
• Usage: Nemotron 3 Ultra leads, MiniMax M3 doubled.
  • Why you should care: usage is the closest thing to a popularity signal the venue publishes; it says nothing about quality, and today's rate limits hit two of the least-used rows.
  • From OpenRouter's free-models collection: Nemotron 3 Ultra leads at 5.36T tokens and ranks 15th in Programming; MiniMax M3 is second at 3.74T, up from 1.88T when the survey last looked, and ranks 20th in Programming; Laguna S 2.1 is third at 1.53T. The page lists 16 entries, one of which is a paid reranker.
• The OpenRouter stealth slot is still empty.
  • Why you should care: if you were waiting for the next free frontier preview, there isn't one; if you were relying on the stealth page to tell you, it won't.
  • The stealth page still presents Ox Alpha as the current stealth model with a 1.05M context window, five days after the row left the catalog. The catalog is the source of truth.

Top models to try

What I'm pointing oxbox at this week. My current workload is still just code reviews of open source projects, but as I get more experience with these models I might branch out.

minimax/minimax-m3:free · OpenRouter · free

Success rate 74% (20 of 27 verified defect findings) · 4 fixes shipped from its verified findings · 98% of completion tokens spent reasoning · 1M context window · free

Twenty of 27 verified findings were real across three batches, and so far the findings it leads with have been the ones that got fixed: four of the defects it reported in oxbox were patched within hours, each commit citing the review. In these runs it labelled uncertainty when it had it and did not invent it when it did not. It also spends 98% of its completion tokens thinking; one 35 KB batch used 84K of oxbox's 100K-token default, so budget for the request rather than the answer. Served by GMICloud behind the OpenRouter route on every run so far.

z-ai/glm-5.3-flash · OpenRouter · cheap paid: $0.075/M in, $0.25/M out

Success rate 72% (63 defect findings, edition 0.2, on the stealth route) · found a real credential leak · $0.075/M in, $0.25/M out · 1.05M context window served · untested on this paid route

Unchanged from the last edition. The revealed Ox Alpha, carrying edition 0.2's 63 confirmed findings at a 72% rate and the credential leak it found in oxbox itself. Nobody has run the paid route yet, so the transfer caveat below still stands.

nemotron-3-ultra-free · OpenCode Zen · free

Success rate 20% (2 of 10 verified defect findings) · severity labels inverted in its one run · a sixth of MiniMax's tokens on the same payload · 129 seconds · free

One run, on the payload matched to MiniMax M3 above: 2 of 10 findings were real, and in that run its severity labels ran opposite to what we found. What it has going for it is cost and hygiene: 5K completion tokens against MiniMax's 31K on the same input, 129 seconds, and every finding formatted exactly as asked. Worth a second run on a different fixture before deciding.

This edition ships a manifest, oxbox-manifest-2026-09-01.json. A manifest is a small JSON file listing the models an issue recommends, in order, with the venue that serves each one, whether it is free or paid, and one line on why. ox --manifest reads it and sends a review to the first entry it is permitted to use, so the survey's advice is machine-readable and pinned to a dated issue instead of retyped from a table. This one carries the three models above in that order, the free MiniMax entry first, so a run with no other flags goes to the model with run evidence rather than skipping past a paid entry to reach it. It is served at oxbox.ai/manifests/latest.json; what a manifest can and cannot do is explained at oxbox.ai/manifests.

What it costs

Cost tradeoffs, in brief

• Free models aren't free to verify. A free model's tokens cost nothing. Every finding it emits costs a reviewer, human or agent, the time to check it against the source, and in our runs so far a false positive has cost as much to refute as a real finding costs to confirm. The money goes on verification, not inference.
• Free routes are paid for in data. Prompts and completions are logged and may be used for training, which is why this pipeline sends only code that is already public.
• Free rows come and go, and are rate-limited: 50 requests a day on an unfunded OpenRouter account. A pipeline that has to finish needs a paid fallback.
• Reasoning-heavy free models spend most of their budget thinking. A token cap sized for the answer truncates them; size it for the request.
• Nothing on a model card has predicted review quality for us yet. Only a run has, and a run costs reviewer time, which is the first point again.

What it cost, both halves

The model's tokens are the cheap half, and on a free route they cost nothing but time. Across the seven MiniMax runs with a response: 45K prompt tokens and 232K completion tokens, 98% of them reasoning (observation); the largest run reported 84.8K reasoning tokens inside 84.0K completion tokens, a subset larger than its set. Nemotron: 7.3K prompt, 4.9K completion, 3.0K reasoning.

The expensive half is the review agent that checks each finding against the source, and it was measured for the matched pair. The two runs produced 36K completion tokens between them. Checking their 22 findings took the review agent 100K output and thinking tokens in one session (60K output, 39K thinking) plus 14M cached context reads, and a further 42K output and thinking tokens in the session that assembled the batches, reproduced the redirect and shipped the fix.

Call it three reviewer tokens generated for every model token, before counting what the reviewer re-read. A false positive costs as much to refute as a real finding costs to confirm, which is why, on this tiny sample, the model at 20% was the more expensive one to use and not only the less useful. The 2026-08-29 batches predate this accounting and carry no reviewer cost.

Relative cost, a rough yardstick

To put the two halves in one currency, this survey prices tokens at Anthropic's published list rates for Claude Opus 5, $5 per million input tokens and $25 per million output, as of 2026-09-01. Opus 5 is the yardstick because it is a strong paid reviewer someone might otherwise send the same code to; nothing here says what Opus would actually have found, and its tokenizer would count the same text differently. The matched pair, one request each on the same 27 KB payload:

The cost-to-check rows spread the pair's total evenly across its 22 findings. That total was about $14.70 in reviewer tokens at list prices, 67 cents per finding: $7.58 from the session that ran the batches, reproduced the redirect and shipped the fix (Opus 5 rates, $6.22 of it cache reads), and $7.11 from the session that verified all 22 (Claude Fable 5.1 rates, $10 and $50 per million with cache reads at $0.25). Against roughly $1 of inference avoided if Opus 5 had produced both reviews at the same token counts, checking cost about fifteen times what the free inference saved. Per real finding, the 20% model cost about three times the 74% one.

Three caveats hold the whole yardstick up. The reviewer figures are an upper bound: both sessions did other work inside the measured windows, and most of the dollars are an agent re-reading its own context every turn. An Opus 5 review would also need checking, though a higher success rate means less to refute. And a human reading the findings instead is not priced at all. The yardstick is meant to be repeated each edition so the ratio has a series to sit in; one pair is a first reading.

Full model list

A listing of what the catalogs show, not a recommendation. Cloaked (or stealth) models are listed under a codename with no lab attached. Named models say who made them. Each priced venue is split into the models that answered one request from this survey's account on 2026-09-01 and the ones that did not, with the reason. That records a moment, not a property; a rate limit says as much about the hour as about the model. The context window is the figure the serving endpoint reports; where a model's card advertises more, the limitations column says so. Which of these got a full review run, and why those, is under "What we ran".

Stealth

The OpenRouter slot is empty. Ox Alpha's exit and reveal were covered in edition 0.3.

big-pickle (OpenCode Zen). Still in the roster, still cloaked, not re-probed by this survey since 2026-08-23, when every call returned 503 from its upstream provider. Reported, not measured: a community evaluation scored it 50.8% on Scale AI's SWE Atlas Codebase QnA (63 of 124 tasks, mini-swe-agent scaffold, 2026-08-11) and says "leaked provider errors and API response signatures suggest it is currently served by DeepSeek infrastructure", while calling the identity "officially unconfirmed" and the result "a snapshot of whatever was behind the alias" that day. Treat the attribution as a guess. This survey's own record on such guesses is one right (Ox Alpha, GLM family); the industry-wide record is poor.

Named — OpenRouter, 21 free

Rate limits are platform-wide: 20 requests per minute, 50 per day, and 1,000 per day after a one-time $10 purchase. "Policy 404" below means the venue answered No endpoints available matching your guardrail restrictions and data policy, a gate set by the calling account's privacy settings rather than a missing model.

Answered, 6 of 21. All six found the planted defect.

Did not answer, 15 of 21.

Named — ZenMux, 5 free

All five are free nowhere else. Free here still needs a funded account: any balance clears the 402 reject_no_credit gate, and calls cost nothing after that.

Answered, 4 of 5. All four found the planted defect.

Did not answer, 1 of 5.

Named — Requesty, 12 free

Five are free nowhere else, and this is the only venue publishing data terms as structured fields. Three of the twelve rows are listings only: their provider does not serve the model.

Answered, 7 of 12.

Did not answer, 5 of 12.

Named — OpenCode Zen, 63 models

The venue publishes no prices, so nothing here is measured as free; see "How far to trust this". Not probed this edition: the roster is unpriced, so a blanket probe could hit paid rows. Confirmed free by earlier calls and still present: nemotron-3-ultra-free (which now has a review run), mimo-v2.5-free, deepseek-v4-flash-free, laguna-s-2.1-free, nemotron-3.5-lightning-free, muse-spark-1.2-contributor-free and the cloaked big-pickle.

What we ran

How the models were chosen. The process the survey is built around is: list every model the venues offer (the catalogs), find the ones that actually answer (the probe), pick a shortlist, and run those in detail against pinned code. This edition did the first two steps for all 38 free rows on the priced venues, and only part of the third.

Seven of the eight detailed runs were oxbox reviewing its own code through the manifest, which sends to the first entry it is permitted to use; with the paid GLM entry skipped, that was MiniMax M3 every time, so those runs were a by-product of oxbox's own development rather than a selection. The one deliberate choice was the Nemotron comparison: the highest-ranked free entry with no run behind it, given a payload MiniMax had already seen.

Thirteen models answered the probe and have never had a run: dots-3-note-preview, ling-3.0-flash-fin, north-mini-code and minimax-m2.7 on OpenRouter; all four ZenMux rows; nemotron-3-super, nemotron-3.5-lightning, gemma-4-31b-it, muse-glimmer-30b and nemotron-3-nano-omni on Requesty. Next edition's shortlist comes from that list, two or three of them on a corpus fixture each.

Eight --mode review runs since the last edition, all against oxbox's own source at pinned commits. Nothing truncated; one call returned no content at all.

MiniMax M3, two batches on 2026-08-29. Four Python files, 15 findings, 13 real (observation). The queue runner and jail test went five for five, including a stale-lock break that spins at full CPU when debris blocks the directory removal, fixed in oxbox sixteen minutes after the run. The exposure gate and preflight script: eight real, one correctly labelled not a defect, one false positive, one true but negligible.

MiniMax M3 against Nemotron 3 Ultra, 2026-08-30. The same two exposure-gate files, the same prompt (MiniMax, Nemotron). MiniMax: 12 findings, 7 real, of which two were fixed eighteen minutes later, one false positive, one wrong mechanism inside a real finding. Nemotron: 10 findings, 2 real, 3 false positives, and the redirect defect filed as UNCERTAIN with a mitigation argument the fix's own reproduction disproves. Nemotron's output followed the format contract exactly, in a sixth of the tokens.

The prompt on 2026-08-30 differed from the 2026-08-29 prompt on the same files by one paragraph saying what the gate was for. MiniMax found the redirect only with that paragraph. Five findings were common to both runs; seven appeared only with the framed prompt and five only without it.

MiniMax M3, three more batches on 2026-08-30, not fully verified. The sandbox tools, the queue runner at a later commit, and ox itself: 15 findings (observation). Six are confirmed by fixes that landed in oxbox within three hours, each commit citing the review; one UNCERTAIN finding about git apply is refuted by git's own documentation; the other eight were not checked, so these batches contribute no rate. From the next edition, a run is reported here only once every finding has been checked.

Availability. One MiniMax call on 2026-08-30 returned no content: 5.6K completion tokens, all reasoning, finish_reason null. One in seven, on a free pool, is a reason for a retry policy and not a verdict.

How far to trust this

• Use your own judgement. I don't vouch for the accuracy of any linked reporting about models. In this survey I will always label things I actually tried myself vs. things that the router or model vendors say; make your own decisions about who to believe.
• One repository, one language, one reviewer. Every run above is Python from oxbox, verified by the person who wrote it. It is the best evidence in the survey and it says nothing about Rust, about a large codebase, or about a reviewer who cannot tell a real finding from a plausible one by reading.
• The prompt is part of the fixture. The same model found the most serious defect in a file only when the task text said what the file was for. A finding count is a property of the model and the prompt together, which is why the survey pins both.
• The cost yardstick is list prices and an upper bound. Reviewer tokens are priced at Anthropic's published rates on the day of writing, the measured windows include other work, and a human reviewer's time is not counted. Read the ratio, not the cents.
• A catalog capture is one moment. A paid-tier change the survey saw on 2026-08-30 had reversed by 2026-09-01, so churn outside the free tier is reported here only after two captures agree.
• A row in a catalog is a listing, not a promise the route answers. Today three Requesty rows and eight OpenRouter rows could not be called, for different reasons, and the Inkling rows are gated to clients this pipeline cannot be. One request at one moment through one account is not a property of the model, and the account's privacy settings are part of what it measures.
• If you see the data-policy 404, check your account, not the model. No endpoints available matching your guardrail restrictions and data policy means your OpenRouter privacy settings exclude every provider serving that row. It is a gate on the caller, and whether looser settings clear it for the eight rows above is untested.
• Reported token counts are not audit-grade. Two of seven MiniMax responses carried usage fields that do not add up, in opposite directions. They are quoted as the provider returned them.
• The router is not the party that saw the code. Every MiniMax run was served by GMICloud behind the OpenRouter route, which the catalog row does not say; the Nemotron response named no provider at all.
• OpenCode Zen publishes no prices. Its catalog returns model IDs and nothing else, so every row's free status is recorded as unknown: not zero, and not inferable from a -free suffix. The only models known to be free there are the ones actually called.
• The free-models collection page and the API disagree. The page lists 16 entries, including a paid VoyageAI reranker, and omits six rows the API returns free (both Gemma 4 rows, both Lyria previews, the content-safety classifier, and the router alias). Its token volumes are a trailing window and move between visits.
• Evidence does not transfer between routes. Edition 0.2's Ox Alpha results came from the stealth route; z-ai/glm-5.3-flash is a different one. Same weights per the reveal; same served behaviour is an assumption until run.

Sources

What this edition drew on, and where to check it.

• The venue catalogs for OpenRouter, ZenMux, Requesty and OpenCode Zen, captured 2026-09-01 UTC and diffed against every capture since 2026-08-24. Every table comes from those, and the full raw catalogs are archived beside them from 2026-08-27 on; the paid MiniMax M3 price in the cost yardstick is read from the same capture.
• The reachability probe: one request per free model on the three priced venues, 2026-09-01, recorded beside each venue's snapshot as an access file. Each reply is kept, trimmed, so a verdict can be argued with.
• The oxbox run logs for the eight review runs, and the five observation files linked above, each of which reproduces the verification table; the two for the matched pair carry the reviewer-token tables the cost yardstick uses.
• Anthropic's pricing page, read 2026-09-01, for the Claude Opus 5 and Claude Fable 5.1 list rates and the prompt-caching multipliers.
• oxbox's commit history for 2026-08-29 and 2026-08-30, in particular the redirect fix, the queue-lock fix, the seeder fix and the scanner fix, each of which names the review that found it.
• OpenRouter's stealth page, which still shows Ox Alpha.
• OpenRouter's free-models collection, the source for the token volumes and category ranks, which the API does not carry.
• OpenRouter's rate-limit documentation.
• OpenRouter's Stripe announcement and Stripe's, both 2026-08-19, for the deal's status and promises. The web searches "OpenRouter Stripe acquisition status September 2026" and "Stripe completes OpenRouter acquisition closed deal" returned only coverage of the agreement, none of a closing.
• The big-pickle SWE Atlas evaluation, found with the search "big-pickle OpenCode Zen stealth model", for the score and the infrastructure guess, quoted with its own caveats.
• Third-party model listings for hy3-free on OpenCode Zen, found with the search "OpenCode Zen hy3-free model", for the "limited time" description; the venue's own roster is the evidence that it left.
• Checked and not used: the search "OpenRouter new stealth model September 2026" returned only Ox Alpha coverage already reported in edition 0.3; the search "MiniMax M3 model release free OpenRouter August 2026" returned pricing pages and a release date of 2026-05-31, nothing bearing on review quality.

---

This edition is published from its source file. The evidence behind it, the catalog snapshots and the manifests, is in oxbox-survey, and nothing from there is edited on the way here.