News Research

Learning to pass: an open problem

We are teaching a policy to pass obstacles and other cars with a staged curriculum. It is not solved yet. Here is where it stands.

Driving a clean lap is one thing. Passing is another. The policy has to see a box or a car in time, pick a side, and commit. We are teaching it in stages, and each stage starts from the best policy of the last one.

Chart of a four-stage passing curriculum with laps and crashes per stage, and the reasons the policy crashed

Four stages

  • Lanes only. 4 laps, 133 crashes on the box test. It drives clean, but it is blind to boxes.
  • Static random boxes. 13 laps, 72 crashes. The basic skill carries over.
  • Moving boxes at 1 m/s. 19 laps, 63 crashes. The most laps and fewest crashes so far.
  • Traffic. Other cars at half to full speed. 19 laps, 24 crashes after 14 minutes of training, but slow: 1.2 m/s.

Why it crashed

  • Blind observations. With 24 rays 8° apart, a 0.3 m box at 2 m fills one ray. The fix: a 48-ray fan, a 10-ray band over 3 m, and a closing rate.
  • Blind reward. "Clear" was the mean of the middle sectors, so a box barely moved it. The fix: clear is the band's minimum.
  • Unfair scale. Paid per meter, a crash cost 1.4 laps. The fix: every lap pays 100 m, and boxes triple from iteration 500.

Where it stands

On wide 2.2 m lanes, a box is a slalom and costs about one crash. On narrow 1.2 m lanes, a box in the middle stops the car: zero laps. The crash rule there leaves about 2 cm of margin. Neither policy has learned to pick a side.

That is the honest state of it. We will post again when it is solved.

All news