News Build log

18 months to our first sim-to-real transfer

NVIDIA thought sim-to-real would be easy. It was not. What broke on the way from a simulated car to a real one, and what we learned.

At GTC 2024, NVIDIA showed a small autonomous race car called Project Leatherback. It was the same event where Isaac Lab was announced. The idea was simple: train a driving policy in simulation, then run it on a real car. The thinking at the time was that sim-to-real would be easy. The software was not ready yet.

GOAT RACER evolution: GR1 July 2025, GR2 December 2025, GR3 September 2026
Three generations: GR1, GR2, GR3.

We took up the challenge. A six-week sprint got the motors spinning. The digital twin and the RL policy took much longer. With several teams and a few long pauses, it was almost 18 months before we had our first real sim-to-real transfer.

The policy that learned to swim

Our first trained policy used just three observations: the cosine and sine of the heading error, and one more. In simulation it looked perfect. On closer inspection, it was exploiting the simulator. It basically learned to swim.

The fixes were not glamorous. We limited joint velocities, added penalties, and stopped training earlier. Train PPO too long and it can start to blow up on itself.

Training in simulation: a generated Laguna Seca track, with speed and slip on screen.

Scaling, shifting, and ONNX

A policy trained in simulation speaks the simulator's units. To run on the car, actions and observations must be scaled and shifted to match the real hardware, and the network must be exported to ONNX for the Jetson. Every one of those steps is a place for a quiet bug.

Not every policy that trains well drives well. Here is one that oscillated on the real car.

Solving physics outside the simulator

For a long time the simulated wheels did not behave like real ones. Rather than distort the car to suit the simulator, we solved the vehicle physics ourselves with a bicycle model in PyTorch, outside Isaac Sim. We called it the magic carpet.

"If I'm going to solve for physics, I'll just solve it outside."

We sent our findings to NVIDIA. Isaac Sim 6 fixed the physics problems, and today we train on NVIDIA Newton.

What we would tell you

  • Start with classic control. If PID cannot drive the car, RL will not either.
  • Do not trust a policy that looks perfect in simulation.
  • Fit the model to the real car, not the other way around.
  • Run 10 or 15 seeds before you claim one policy is better than another.

Sim to real on GR2, June 2026.

That 18 months is what our IP package lets a team skip.

Draft note: confirm the third observation in the first policy.

All news