What we found
01 / Argon is a step change on robot data
On held-out assembly video it labels actions with the right verb, part and timing at F1 0.58, against 0.11 for Gemini 3.1 Pro and 0.09 for Gemini 3 Flash on identical inputs (gap over 3.1 Pro +0.39 to +0.56, 95% interval). Judging real success, it reaches AUROC 0.89 against 0.76 for 3.1 Pro (gap +0.06 to +0.20) and 0.52 for 2.5 Pro; 0.997 on pick and place.
02 / Failed retries leak through
With our labeling prompt on correctly timed clips, Argon labels 36 to 52% of human-marked failed-attempt time as success over three fresh runs, and merges 6 to 11 of 32 failed attempts into the successful retry that follows (Figure 2). Only 1.6 to 2.7% of successful time is wrongly flagged, so the error runs one way: if these labels decide which segments behavior cloning down-weights, as the launch describes, many failures keep full weight.
Model re-checks we tried added many false alarms; robot signals close part of the gap with no model calls: flagging empty grasps, lift-and-retry and same-spot re-grasps raised failure recall from 0.57 to 0.68 and cut failed time kept as success from 33% to 22%, short of our pre-registered 0.70 recall target.
03 / Argon calls unfinished work done
On frames from just before a successful action finished, it said done 33 to 39% of the time at every thinking level (91% for parts grasped but not yet lifted): a reward-hacking surface for progress and value estimates. A one-line completion definition removes it (2 of 322) but fails more real successes (52 vs 35 of 308); a protocol Argon wrote for itself gets 30 of 322 while failing fewer (19 of 308).
04 / Robot state fixes what pixels cannot
With no extra model calls, snapping boundaries to gripper open/close events halves timing error (1.33 to 0.72 s). The gripper’s closed width gauges part size (about 2.5, 7, 11 and 15 mm for pegs 1 to 4), fixing Argon’s main naming error (90% to 96% of parts right). Gripper-derived primitives more than double primitive F1 (0.31 to 0.72). On correctly timed clips the same step reaches action F1 0.79.
05 / Argon’s timestamps follow the clip’s clock
That exposed an error in ours. The wrist camera records at about 28 Hz but its files say 30, so our wrist clips played 7% fast and Argon’s wrist-only times drifted 0.054 s per second (the clocks predict 0.068). A fresh pre-registered run on clips played at true speed cut the drift to 0.008 s per second and raised action F1 from 0.48 to 0.74 for the wrist camera alone, and from 0.58 to 0.68 for both cameras at 2 frames per second. At equal tokens the wrist camera alone matches both cameras at 1 frame per second (0.74 vs 0.73); the external camera alone names 27% of parts.
06 / Inputs move accuracy; prompts do not
Ten judging prompt variants (thinking levels, self-written protocols, reference examples, force and sound plots) land at AUROC 0.87 to 0.91, while one or two cameras instead of three give 0.81 to 0.84. With both cameras, labeling accuracy also fades across each two-minute clip (0.83 in the first 30 s, 0.71 at 60 to 90 s); the wrist camera alone holds steady (0.73 to 0.79), and gripper snapping recovers most of the loss. One frame per second matches two at half the tokens.
Evidence
Hindsight labeling
| Labeler | Action F1 | Structure F1 | Parts right | Boundary error | Failed attempts caught | Primitive F1 | Tokens per video hour |
|---|---|---|---|---|---|---|---|
| Same clips for every model (wrist clip 7% fast) | |||||||
| Gemini 3 Flash, 2 fps | 0.09 | 0.20 | 43% | 1.71 s | 0% | 0.22 | 1.0 M |
| Gemini 3.1 Pro, 2 fps | 0.11 | 0.23 | 46% | 1.64 s | 3% | 0.23 | 1.2 M |
| Argon, 2 fps | 0.58 | 0.69 | 84% | 1.34 s | 43% | 0.27 | 1.1 M |
| Argon, 1 fps | 0.71 | 0.78 | 90% | 1.33 s | 32% | 0.31 | 0.57 M |
| Argon, 1 fps + signals | 0.75 | 0.79 | 96% | 0.72 s | 32% | 0.72 | 0.57 M |
| Argon on correctly timed clips (one fresh pre-registered run each) | |||||||
| Both cameras, 2 fps | 0.68 | 0.79 | 85% | 1.50 s | 54% | 0.34 | 1.15 M |
| Both cameras, 1 fps | 0.73 | 0.82 | 90% | 1.35 s | 51% | 0.33 | 0.59 M |
| Both cameras, 1 fps + signals | 0.79 | 0.85 | 93% | 0.74 s | 57% | 0.74 | 0.59 M |
| Wrist camera only, 2 fps | 0.74 | 0.83 | 89% | 1.23 s | 43% | 0.36 | 0.62 M |
| Wrist camera only, 2 fps + signals | 0.78 | 0.84 | 93% | 0.71 s | 51% | 0.73 | 0.62 M |
Signals: the robot-signal pipeline (gripper snapping, sizing, primitives). Structure F1 requires the right verb and timing; parts are scored on those matched actions; failed attempts caught counts human-marked failures matched to an Argon failure flag. Repeat 1 fps runs on the earlier clips scored 0.64, 0.66 and 0.71 action F1, and the same frozen robot-signal step added +0.05, +0.08 and +0.05 (95% interval of the pooled gain +0.03 to +0.08); on correctly timed clips it added +0.05 at 1 fps and +0.11 at 2 fps.
Failed-retry check from robot signals
| Argon run (both cameras unless noted) | Failed attempts caught | Precision of failure flags | Failed time kept as success | Successful time flagged |
|---|---|---|---|---|
| 1 fps, true clock (primary) | 0.57 → 0.68 | 0.81 → 0.66 | 33% → 22% | 2.7% → 3.5% |
| Wrist camera only, 2 fps, true clock | 0.51 → 0.54 | 0.79 → 0.61 | 43% → 39% | 1.6% → 3.1% |
| 2 fps, true clock | 0.51 → 0.62 | 0.73 → 0.62 | 50% → 35% | 2.2% → 3.8% |
| 1 fps, earlier clips, run 1 | 0.32 → 0.46 | 0.86 → 0.71 | 60% → 41% | 1.4% → 2.6% |
| 1 fps, earlier clips, run 2 | 0.22 → 0.41 | 0.67 → 0.62 | 68% → 48% | 1.3% → 3.1% |
| 1 fps, earlier clips, run 3 | 0.43 → 0.62 | 0.73 → 0.66 | 54% → 35% | 1.3% → 2.4% |
Argon plus the robot-signal pipeline, without and with the check (before → after). The check flags an empty grasp, a lift-and-retry while holding the part, or a same-spot re-grasp after an insert, and splits the action there; its rules were chosen on the development recordings and frozen before this run.
Pre-registered results: recall at least 0.70 at precision at least 0.60 (missed: 0.68, 95% interval 0.48 to 0.83, at 0.66); more catches than without it (missed: 4 vs 0, p 0.13); failed time down at least 10 points with successful time up at most 3 (met); recall up on all six runs (met). Across the six runs it added 28 catches and lost none; the runs share recordings, so they are not independent.
Success judging on real failures
| Judge | REASSEMBLE assembly (AUROC) | FailBench pick and place (AUROC) | Certified verdicts, FailBench (all 140 test steps) |
|---|---|---|---|
| Gemini 4 Argon | 0.89 | 0.997 | 129 of 140 at a 5% error target (2.3% wrong) |
| Gemini 3.1 Pro (preview) | 0.76 | 0.88 | Not tested |
| Gemini 3 Flash | 0.67 | 0.88 | None at 5% or 10% |
| Gemini 2.5 Pro | 0.52 | 0.72 | Not tested |
Argon’s lead over 3.1 Pro on REASSEMBLE is +0.13 AUROC (95% interval +0.06 to +0.20, resampling whole recordings). On FailBench’s own pick-and-place protocol (balanced accuracy, all 140 test steps) our Gemini 3 Flash scores 0.83, matching the published 0.84; Argon scores 0.993. Certification uses Learn-then-Test on held-out calibration tasks.
Separately, cases built only from successful runs (wrong instruction, steps played backwards, near-complete frames) rank 12 related judge configurations like real failures do (rank correlation 0.87 to 0.96 on REASSEMBLE, 0.95 on FailBench pick and place); across six distinct models the REASSEMBLE ranking is weaker (0.54), and none of these cases gives a judge’s absolute accuracy. In replay on our saved judgments, the exact comparison engine certified the best of the 12 configurations with 35% (REASSEMBLE) and 59% (FailBench) fewer judgments than scoring everything, with every winner checked independently.
What this suggests for Argon
- Feed proprioception into hindsight labeling. Gripper width and events carry timing and identity that video does not.
- Benchmark failed-retry QC as its own capability, and give it robot events, before its labels weight training data.
- Ship a completion definition with success and progress prompts, or train on near-complete negatives.
- Check clocks, then cameras, then prompts. Re-time mixed-rate cameras before labels are used; timed correctly, the wrist camera alone matches two. Label in shorter or overlapping windows, or anchor them to robot events.
Next test: evaluation on actual deployment data. Score Argon’s labels and verdicts against operator takeovers in the field and measure how much failed time would reach training.
How the work was done
Pre-registered
Hypotheses, splits and pass/fail rules were frozen in hashed records before grading. Results are reported whether or not a test passed: the robot-signal step’s pre-registered single-run gain (+0.045) narrowly missed its +0.05 bar; a per-action failure re-check failed its precision rule; the six-model ranking missed its 0.70 bar; the robot-signal retry check missed its 0.70 recall target and its paired test.
Held out
Fixes were designed on 12 development recordings and tested on 12 others. The 1 fps setting was chosen on held-out runs and then confirmed on a fresh run frozen in advance. Model gaps carry intervals that resample whole recordings.
Audited from the inputs up
A separate re-analysis of the saved model outputs caught two errors in our own pipeline, both fixed here: structure matches did not require the same verb, and wrist clips played 7% fast (finding 5). The second changed two conclusions: the wrist camera, not the external one, carries the action structure, and the failed-retry leak is smaller than the mistimed clips showed (36 to 52% of failed time, not 48 to 68%). A fresh pre-registered run on corrected clips confirmed the direction; of its four predictions, one passed, two failed because scores came in above the predicted range, and one failed because a small wrist-only drift remains.
Public data, reproducible
REASSEMBLE (CC BY 4.0) and FailBench (Apache-2.0). All models were called through Vertex AI with identical prompts. Code, protocols and raw model outputs are available on request.
Limitations
- Hindsight labeling was measured on one robot cell (REASSEMBLE) with one parallel gripper; 32 to 37 failed attempts in the held-out set. Evaluation on actual deployments is the next test.
- Results on correctly timed clips are one fresh run per setting; earlier repeat runs varied by up to 0.07 action F1. The other Gemini models were compared only on the earlier clips, which every model saw identically.
- Both datasets were public before Argon’s release; training exposure cannot be ruled out. Fresh data from actual deployments removes this concern.
- Prompts listed the task’s verbs and parts. The gripper size catalogue came from labeled development recordings.
- The failed-retry finding uses our own labeling prompt with success flags; Google’s internal quality-control prompt may behave differently. Both it and the early-done finding are measured on recordings, not on a policy trained against Argon.
- The dataset’s own labels are imperfect: in 6 actions the measured grasp width contradicts the human part label and Argon agrees with the width. Scores keep the human labels as given.