AutoAgent0

01 / OVERVIEW

Safe autonomous driving requires more than accurate planning at a single frame: it demands reliable closed-loop execution in dynamic environments. Accumulated errors, especially in unseen scenarios, can push a vehicle into states from which the same policy struggles to recover.

We introduce AutoAgent0, an agentic runtime and harness framework that augments existing driving experts with complementary detection skills, rule-based action verification, and feedback-driven recovery. An independent detection module constructs scene context, while a shared verifier assesses trajectory feasibility before execution.

When nominal trajectories become unreliable, Recovery Planning constructs alternative maneuvers from action primitives and iteratively refines them using verification feedback. Evaluations on Fail2Drive and NavSafe-∞ show improved closed-loop robustness across multiple frozen driving experts. On NavSafe-∞, recovery improves task success at the cost of lower comfort and driving efficiency.

02 / EMPIRICAL RESULTS

LONG-TAIL SUCCESS RATE81.0%

+28.3 pp over SparseDriveV2

AVERAGE SUCCESS RATE90.3%

BridgeDrive + AutoAgent0

FAIL2DRIVE EXPERTS3 / 3

Improved on both evaluation splits

Benchmark Results on Fail2Drive

DS: driving score. SR: success rate. HM combines DS and SR using their harmonic mean. Average combines both splits; parentheses show gains over each standalone expert.
Method	Inputs		In-Distribution			Generalization			Average
Method	RGB	LiDAR	DS ↑	SR (%) ↑	HM ↑	DS ↑	SR (%) ↑	HM ↑	DS ↑	SR (%) ↑	HM ↑
TCP	✓	—	24.7	39.1	30.3	24.5	31.4	27.5	24.6	35.3	28.9
Orion	✓	—	53.0	52.0	52.5	51.2	46.0	48.5	52.1	49.0	50.5
HiP-AD	✓	—	74.1	70.7	72.4	67.1	56.7	61.5	70.6	63.7	67.0
SimLingo	✓	—	82.6	79.3	80.9	71.7	55.0	62.2	77.2	67.2	71.6
AlignDrive	✓	—	74.7	75.7	75.2	68.6	62.7	65.5	71.7	69.2	70.4
BevAD	✓	—	87.4	83.3	85.3	82.3	68.7	74.9	84.9	76.0	80.1
TF++	✓	✓	83.3	78.5	80.8	75.4	61.1	67.5	79.4	69.8	74.2
TFv6	✓	✓	90.2	93.3	91.7	79.5	70.7	74.8	84.9	82.0	83.3
UniAD	✓	—	47.5	36.3	41.2	44.0	27.6	33.9	45.8	32.0	37.6
UniAD + AutoAgent0	✓	✓	68.4	69.0	68.7	69.2	69.0	69.1	68.8(+23.0)	69.0(+37.0)	68.9(+31.3)
SparseDriveV2	✓	—	70.0	74.0	71.9	62.9	52.7	57.4	66.5	63.4	64.7
SparseDriveV2 + AutoAgent0	✓	✓	80.7	85.7	83.1	79.2	81.0	80.1	80.0(+13.5)	83.4(+20.0)	81.7(+17.0)
BridgeDrive	✓	✓	91.6	95.0	93.3	81.9	75.0	78.3	86.8	85.0	85.8
BridgeDrive + AutoAgent0	✓	✓	91.7	95.3	93.5	85.7	85.3	85.5	88.7(+1.9)	90.3(+5.3)	89.5(+3.7)

FAIL2DRIVE

Performance Comparison

Higher is better ↑

Metric

Standalone expert+ AutoAgent0

200 routes in 100 matched pairs, covering 17 long-tail scenario types. Results are averaged over three runs.

NavSafe-∞ evaluation 280 scenarios

Results averaged over three runs. DS: driving score; SR: success rate (%); DE: driving efficiency. Higher is better for all scores; parentheses show gains over the standalone expert.
Method	DS	SR	DE	Comfort	TC	VRUC	TV	TI
DrivoR	64.7	50.7	188.0	87.9	58.7	31.7	52.4	43.3
GTRS-Dense–V2-99	64.3	49.5	163.9	95.9	52.0	40.0	54.2	36.7
AutoVLA	51.1	29.5	186.0	65.4	33.0	15.0	29.7	36.7
RAP	47.7	27.3	220.8	48.4	32.3	14.2	28.8	22.2
MTDrive–SFT	48.8	31.8	173.8	94.5	30.3	15.0	35.8	44.4
LTF	53.0	27.7	163.7	97.8	31.7	25.0	28.5	15.6
LTF + AutoAgent0	74.8(+21.8)	60.5(+32.8)	160.0	78.8	66.0(+34.3)	45.8(+20.8)	63.0(+34.5)	52.2(+36.6)
DiffusionDrive	44.8	21.0	165.1	97.7	21.3	11.7	23.6	22.2
DiffusionDrive + AutoAgent0	70.9(+26.1)	52.3(+31.3)	160.2	72.3	52.0(+30.7)	37.5(+25.8)	58.8(+35.2)	48.9(+26.7)
SparseDriveV2	65.3	48.8	168.0	93.4	54.0	43.3	48.2	41.1
SparseDriveV2 + AutoAgent0	77.7(+12.4)	63.6(+14.8)	165.3	76.0	68.0(+14.0)	50.0(+6.7)	64.5(+16.3)	63.3(+22.2)

TC: traffic crashes · VRUC: vulnerable road user crashes · TV: traffic violations · TI: traffic incidents. All three augmented experts use BridgeDrive’s native perception; the additional trained perception specialists are used only in Fail2Drive. Higher success comes with lower comfort and efficiency.

03 / CLOSED-LOOP DEMONSTRATIONS

Fail2Drive Demonstrations

01 / 04

NavSafe-∞ Demonstrations

SparseDriveV2 and SparseDriveV2 + AutoAgent0, compared in scene and bird’s-eye views.

Runtime Demonstrations

04 / METHOD

AutoAgent0 architecture: environment observations, nominal planning, scene context refinement, a rule-based verifier, and recovery planning. — **The AutoAgent0 harness.** Independent scene context supports a rule-based verifier. Nominal and recovery trajectories pass through the same admission checks before reaching the tracking controller.

Agentic Recovery Example

EXECUTION RULE

Admitted nominal plan→Execute expert trajectory

No nominal plan admitted→Search, refine & verify recovery

If no recovery candidate is admitted before the next control decision, use conservative braking.

05 / MORE ANALYSIS

GENERALIZATION GAP

Less degradation on unfamiliar routes.

UniAD3.5 → −0.8

SparseDriveV27.1 → 1.5

BridgeDrive9.7 → 6.0

The driving-score gap between familiar and unfamiliar routes narrows with AutoAgent0, while scores improve on both.

RUNTIME COMPARISON

Recovering route progress.

85.0%Success on unfamiliar routes
with AutoAgent0

With the same SparseDriveV2 expert, AutoAgent0 achieves higher driving scores and success rates than six alternative runtimes on both familiar and unfamiliar routes.

160 evaluation routes, with one run per route.

Comparing runtime approaches 160 routes

Runtime	ID DS ↑	ID SR (%) ↑	Gen. DS ↑	Gen. SR (%) ↑
SparseDriveV2	70.3	75.0	63.1	52.5
+ RSS	73.5	66.2	65.5	45.0
+ Longitudinal MPC	69.9	55.0	65.4	42.5
+ 2D MPC	65.2	65.0	60.1	47.5
+ Uncertainty monitor	74.5	60.0	60.7	32.5
+ AESOP (adapted)	56.3	52.5	45.7	45.0
+ AESOP + primitives	67.3	62.5	57.8	60.0
+ AutoAgent0	79.8	82.5	81.6	85.0

What each runtime component adds SparseDriveV2

Verification reduces collisions. Better scene context and recovery planning help the vehicle finish more routes, though recovery introduces more collisions than conservative fallback.

Variant	Context refinement	Rule-based verification	Recovery planning	DS ↑	SR (%) ↑	Col. ↓	Rec./km
SparseDriveV2	—	—	—	66.5	63.4	0.445	0.0
+ Rule-based verification	—	✓	—	69.1	60.0	0.095	—
+ Scene context refinement	✓	✓	—	76.0	74.0	0.050	54.2
+ Recovery planning (full)	✓	✓	✓	80.0	83.4	0.145	20.3
AutoAgent0 with ground-truth context	GT	✓	✓	81.4	90.3	0.110	26.2

What makes recovery effective SparseDriveV2

Choosing maneuvers to fit the scene and revising them with verifier feedback improves recovery. Compared with retries using the same budget, feedback helps finish more routes but also leads to more collisions.

Variant	DS ↑	SR (%) ↑	Col. ↓	Rec.-SR (%) ↑
Controlled fallback	76.0	74.0	0.050	—
+ Random primitive selection	74.0	73.0	0.210	52.7
+ VLM skill selection	76.3	77.0	0.125	61.7
+ Parameterized recovery	76.3	77.3	0.200	62.5
+ Budget-matched retry	77.8	77.3	0.125	63.5
+ Verifier-guided refinement	80.0	83.4	0.145	67.4

Improving obstacle detection 20 episodes

Specialized detectors improve pedestrian and static-obstacle detection across 3,587 frames, while vehicle predictions stay unchanged. These specialists are used in Fail2Drive; NavSafe-∞ uses BridgeDrive's native perception.

Category / system	Score threshold	TP	FP	P (%) ↑	R (%) ↑	F1 (%) ↑
Vehicle / LEAD	0.35	17,046	1,195	93.45	92.80	93.12
Vehicle / + specialists	0.35	17,046	1,195	93.45	92.80	93.12
Pedestrian / LEAD	0.35	0	15	0.00	0.00	0.00
Pedestrian / + specialists	0.10	637	298	68.13	60.49	64.08
Pedestrian / + specialists (reference)	0.35	573	88	86.69	54.42	66.86
Static / LEAD	0.35	662	2,144	23.59	11.06	15.06
Static / + specialists	0.35	3,177	2,816	53.01	53.10	53.06

06 / CITATION

@misc{autoagent0,
  title = {{AutoAgent0}: An Agentic Runtime for Safe Closed-Loop Driving},
  author = {Liu, Jiageng and Zhao, Seth Z. and Zhou, Zewei and Huerta, Alejandro and Bayer-Calvert, Aidan and Bao, Yuxin and Lin, Haotian and Spataru, Robert and Cui, Zijun and Hong, Yining and Zhou, Bolei and Ma, Jiaqi},
  year = {2026},
  eprint = {2610.18866},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2610.18866}
}