Nvidia open-sources physical AI tools for robotics

NVIDIA Open Sources Physical AI Agent Tools for Robotics and Autonomous Vehicles

NVIDIA announced a major collection of open-source physical AI agent skills and tools at GTC Taipei on June 1, enabling AI coding agents to directly orchestrate complex robotics, autonomous vehicle, vision AI, and industrial digital twin workflows.nvidia

Turning the Physical AI Stack Into Agent-Callable Tools

The release, available through GitHub and skills.sh, packages NVIDIA’s physical AI libraries as agent-executable tasks within the NVIDIA Agent Toolkit. The tools span Cosmos world foundation models, Omniverse simulation libraries, Isaac for robotics, Metropolis for vision AI, Alpamayo for autonomous driving, and the Jetson edge AI platform.globenewswire

“AI agents are revolutionizing software development, and that shift is now coming to physical AI, extending into the systems that will transform transportation, manufacturing, healthcare and robotics,” CEO Jensen Huang said during his keynote. “When agents can directly use NVIDIA libraries, models and frameworks, physical AI development will move faster, enabling developers to build the robots, autonomous vehicles and industrial systems of the future at an incredible pace.”nvidia

Industry Adoption and Use Cases

The tools convert complex development pipelines — data generation, simulation, training, evaluation, and deployment — into repeatable instructions that coding agents can follow. NVIDIA said developers can also deploy autonomous agents securely using the NemoClaw blueprint and OpenShell runtime, which enforces policy-based security and privacy guardrails.nvidia

Industry partners are already applying the technology. TSMC and Pegatron are using the tools for visual inspection model fine-tuning, with Pegatron reporting a 67% reduction in model training and deployment time through synthetic data generation. Li Auto, Afari, and DeepRoute.ai are generating more than 300,000 renders and simulations per day using Omniverse neural scene reconstruction for autonomous driving development.globenewswire

Broader Ecosystem

Cadence, Dassault Systèmes, Siemens, and Synopsys are integrating the tools for engineering simulation and digital twins, while Microsoft, CoreWeave, and Nebius are offering cloud integrations for scaling synthetic data generation. Robotics firms including 1x, Agile Robots, Agility, and Universal Robots are using the agent-ready stack across their development pipelines.nvidia

The announcement extends NVIDIA’s push to make physical AI development more accessible, building on the Physical AI Data Factory Blueprint and expanded open models unveiled at GTC in March.nvidia