The Oxbox Survey
Which cheap or cloaked model is worth trying this week
Free and stealth models come and go weekly: anonymous, free, suspiciously capable. Or perhaps you just want to run a cheap model that you don't completely trust. The Oxbox Survey tries them on real code from inside a sandbox, and says how far each number can be trusted.
[PASS]network: outbound TCP to 1.1.1.1:443(PermissionError)
[PASS]network: DNS resolution(gaierror)
[PASS]fs read: ~/.ssh(PermissionError)
[PASS]fs metadata: stat ~/.ssh (oracle)(PermissionError)
[PASS]env: parent environment not inherited(succeeded)
…
jail holds: 14/14 probes passed
[PASS]oxseed refuses parent traversal
[PASS]oxbox refuses --work outside sandbox/
[PASS]oxbox refuses stdout redirected outside the sandbox
[PASS]oxapply refuses traversal in rename headers
[PASS]ox refuses a key in the task argument
…
guards hold: 22/22 passed
How a model gets rated
Every claim carries one of three evidence labels. Measured is a catalog capture you can re-run. Observed is what happened in an oxbox run, small-sample and specific to one reviewer's repos. Reported is the vendor's word, not verified. Every model the survey has actually run gets three measured 0 to 5 digits for quality, cost, and speed, a note of anything that disqualifies it, and an Editor's Rating of Good, Acceptable, Marginal, or Poor. A model nobody has run is never rated. The rules in full.
The jail the runs happen in
The runs use oxbox: a supervisor that lets an untrusted model see only the files you hand it, apply its patch only into a sandbox, and executes any changed code with no network and no way out. It records exactly what was sent, and where. The tool, its documentation, and its tests live on GitHub.
Each issue ships a manifest of the week's picks that oxbox reads directly, so you can use the survey's choice directly if you want.