commaai / openpilot

openpilot is an operating system for robotics. Currently, it upgrades the driver assistance system on 275+ supported cars.
https://comma.ai/openpilot
MIT License
49.98k stars 9.11k forks source link

Metadrive: Flaky test behavior with Dragon Rider #34044

Open JakobSpahn opened 4 days ago

JakobSpahn commented 4 days ago

Describe the bug

ONNXCPU=1 pytest tools/sim/test/test_metadrive_bridge.py fails locally when using Dragon Rider.

FAILED tools/sim/tests/test_metadrive_bridge.py::TestMetaDriveBridge::test_driving - AssertionError: Simulator fails to finish a loop. Failure states: ['out_of_lane']

When switching to the previous model, the test always succeeds.

It would be very informative if anyone manages to reproduce (or not reproduce).


Switched between the following models:

Dragon Rider

git checkout 47bd0f0166db1268bfa5504d7b95f94264d15500 -- selfdrive/modeld/models/supercombo.onnx && scons -u -j$(nproc)

MLSIM V4

git checkout 34dde0bb36b528febee292891523e30e82835aac -- selfdrive/modeld/models/supercombo.onnx && scons -u -j$(nproc)

OS Version

Ubuntu 24.04

openpilot version or commit

c11e9a3bdd7cd490c9243d8cdf90ef8af64fd9ba

Additional info

No response