Motional has released nuReasoning, an open dataset of 20,000 long-tail driving scenarios with 247,000 human-verified reasoning annotations intended to train autonomous vehicles in human-like decision-making. The company describes nuReasoning as the first reasoning-centric, long-tail open dataset for AVs, built to supply end-to-end autonomy systems with the logic behind driving actions rather than the actions alone. Motional developed the dataset and a companion research challenge with the UCLA Mobility Lab and its director, Professor Jiaqi Ma.
Highlights
- 20,000 long-tail scenarios, each a video clip of at least 20 seconds selected from Motional’s fleet driving data
- 247,000 reasoning annotations with human-verified quality assurance across spatial, decision, and counterfactual reasoning
- More than 105 hours of edge-case driving mined across Las Vegas, Pittsburgh, Los Angeles, Boston, and Singapore
- nuReasoning Challenge launching at ECCV in Sweden, with winners announced at NeurIPS in December
Why Reasoning Annotations Matter for Vision-Language-Action Models
According to Motional, current AI is moving beyond recognizing objects toward understanding the physical world, anticipating what happens next, and deciding how to act. Vision-Language-Action (VLA) models combine those capabilities, but the company says they require training data that conveys the reasoning behind a correct action, not just the action itself.
nuReasoning is designed to fill that gap by teaching models to interpret spatial relationships, reason through driving decisions, anticipate risk, and weigh alternative outcomes. Each annotated event lets researchers see not only what the AV perceived, but why it took a given action and why alternatives were judged unsafe.
Motional offers one example: a nighttime construction-zone clip in which the AV stops before proceeding. Without context, the stop could read as the vehicle failing to understand the work zone. The annotation shows the AV stopped correctly for a small animal crossing ahead, and that alternate paths were rejected because of construction barriers and the animal’s unknown speed and direction.
A nuReasoning miniset released earlier this year has been downloaded more than 50,000 times, according to the company.
What Does the nuReasoning Dataset Contain?
Mined from Motional fleet data across five cities, the dataset includes:
- Over 105 hours of reasoning-intensive edge cases, including unusual pedestrian activity, work zones and night road construction, animal crossings, and limited-visibility scenarios
- Three annotation types supporting VLA model training: Spatial Reasoning, Decision Reasoning, and Counterfactual Reasoning
- 247,000 reasoning annotations with human-verified quality assurance
- Multi-modal sensor data providing full scene representation in 3D space
Motional has also integrated its proprietary Omnitag data search engine, which lets researchers query dataset distributions by scenario type, difficulty level, and location. Omnitag supports natural-language semantic search, so users can enter a query such as “emergency vehicle approaching behind with a construction site nearby” and isolate matching clips.
“To safely expand AV fleets, autonomous driving systems must react to rare, chaotic edge cases with the same assured logic as an experienced human driver,” said Phil Michel, Motional’s Senior Vice President of Autonomy and AI. “At Motional, we’re prioritizing transparent AI so we can clearly evaluate real-time decision-making and risk assessment. By making nuReasoning openly available, we’re offering a shared foundation to help the entire industry solve edge cases and advance toward scalable autonomous operation.”
nuReasoning Challenge at ECCV
Motional and the UCLA Mobility Lab are hosting the nuReasoning Challenge, launching at the European Conference on Computer Vision (ECCV) in Sweden. The challenge is structured as a joint evaluation of planning and reasoning on 1,000 private-test scenarios, across two tracks:
- Explainable Trajectory & Motion Planning: benchmarking motion-planning accuracy and safety compliance when guided by explicit counterfactual reasoning
- Long-Tail Visual Question Answering & Scene Reasoning: evaluating a model’s ability to infer spatial relationships, causal decision traces, and risk factors in edge cases
Winners will be announced at the Conference on Neural Information Processing Systems (NeurIPS) in December.
From nuScenes to nuReasoning
nuReasoning extends Motional’s open-data lineage, which began with nuScenes in 2019, described by the company as the industry’s first multi-modal public AV dataset, and continued through nuImages, Panoptic nuScenes, and nuPlan. Motional says the open benchmarks are intended to help academic and commercial engineers establish shared safety standards for AV perception, trajectory planning, and explainable AI.
The datasets are available for commercial licensing or free academic use under Motional’s non-commercial terms. Motional, majority-owned by Hyundai Motor Group, currently offers robotaxi service on the Uber network in Las Vegas and is targeting driverless operations there by the end of 2026.
