commaai / openpilot

openpilot is an operating system for robotics. Currently, it upgrades the driver assistance system on 275+ supported cars.
https://comma.ai/openpilot
MIT License
49.89k stars 9.09k forks source link

Config_realtime_process crashes in modeld_replay.py on C3X #33625

Closed haraschax closed 1 month ago

haraschax commented 1 month ago

Describe the bug

Just running model_replay.py on C3X:

comma@comma-8fcff1ab:/data/openpilot/selfdrive/test/process_replay$ ./model_replay.py 
Starting listener for: camerad
Using cached CarParams
VIN 00000000000000000
{'event': 'fingerprinted', 'car_fingerprint': 'RAM_1500_5TH_GEN', 'source': <FingerprintSource.fw: 'fw'>, 'fuzzy': False, 'cached': True, 'fw_count': 8, 'ecu_responses': [], 'vin_rx_addr': -1, 'vin_rx_bus': -1, 'fingerprints': '{0: {173: 5, 49: 8, 125: 6, 55: 8, 317: 8, 35: 8, 149: 7, 941: 8, 53: 8, 171: 8, 43: 8, 47: 8, 113: 8, 37: 8, 121: 8, 41: 8, 127: 8, 131: 8, 155: 8, 179: 8, 181: 8, 39: 8, 213: 3, 141: 8, 221: 8, 299: 8, 319: 8, 164: 8, 169: 8, 323: 8, 177: 3, 343: 8, 133: 8, 147: 8, 123: 7, 409: 6, 119: 2, 135: 8, 137: 8, 139: 8, 157: 8, 167: 8, 145: 8, 129: 8, 347: 8, 676: 8, 307: 8, 295: 8, 992: 8, 311: 8, 333: 8, 623: 8, 297: 4, 754: 8, 940: 8, 293: 3, 650: 8, 818: 8, 298: 8, 315: 8, 649: 8, 334: 8, 792: 8, 421: 8, 657: 8, 1031: 8, 993: 7, 296: 8, 645: 2, 969: 4, 856: 4, 289: 5, 634: 8, 643: 8, 860: 6, 939: 8, 641: 8, 635: 8, 673: 8, 557: 8, 938: 8, 663: 8, 848: 8, 503: 8, 609: 8, 689: 5, 505: 8, 611: 8, 448: 6, 633: 8, 1026: 8, 584: 8, 560: 8, 605: 8, 1003: 8, 607: 8, 875: 2, 456: 4, 659: 5, 1000: 8, 937: 8, 601: 8, 613: 8, 752: 2, 1025: 8, 720: 6, 491: 8, 825: 2, 687: 8, 631: 8, 489: 8, 507: 5, 502: 8, 679: 8, 773: 8, 1024: 8, 1062: 8, 1059: 8, 1050: 8, 1033: 8, 464: 8, 943: 8, 942: 8, 897: 8, 710: 8, 709: 8, 706: 8, 973: 8, 1014: 8, 979: 8, 980: 8, 982: 8, 981: 8, 1013: 8, 983: 8, 1012: 8, 984: 8, 1011: 8, 962: 8, 1009: 8, 1100: 8, 1010: 8, 1015: 8, 948: 8, 947: 8, 1008: 8, 1098: 8, 926: 3, 1002: 8, 1001: 8, 910: 8, 995: 8, 996: 8, 906: 8}, 1: {}, 2: {163: 8, 153: 8, 166: 8, 232: 8, 250: 8, 838: 2}, 3: {}, 4: {}, 5: {}, 6: {}, 7: {}}', 'fw_query_time': 0.010975954000059573}
modeld init
crash
Traceback (most recent call last):
  File "/data/openpilot/selfdrive/modeld/modeld.py", line 292, in <module>
    main(demo=args.demo)
  File "/data/openpilot/selfdrive/modeld/modeld.py", line 121, in main
    config_realtime_process(7, 54)
  File "/data/pythonpath/openpilot/common/realtime.py", line 44, in config_realtime_process
    set_core_affinity(c)
  File "/data/pythonpath/openpilot/common/realtime.py", line 37, in set_core_affinity
    os.sched_setaffinity(0, cores)
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
  File "/data/openpilot/selfdrive/modeld/modeld.py", line 292, in <module>
    main(demo=args.demo)
  File "/data/openpilot/selfdrive/modeld/modeld.py", line 121, in main
    config_realtime_process(7, 54)
  File "/data/pythonpath/openpilot/common/realtime.py", line 44, in config_realtime_process
    set_core_affinity(c)
  File "/data/pythonpath/openpilot/common/realtime.py", line 37, in set_core_affinity
    os.sched_setaffinity(0, cores)
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
  File "/data/openpilot/selfdrive/test/process_replay/./model_replay.py", line 86, in <module>
    log_msgs += model_replay(lr, frs)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/data/openpilot/selfdrive/test/process_replay/./model_replay.py", line 65, in model_replay
    modeld_msgs = replay_process(modeld, modeld_logs, frs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/pythonpath/openpilot/selfdrive/test/process_replay/process_replay.py", line 666, in replay_process
    process_logs = _replay_multi_process(cfgs, all_msgs, frs, fingerprint, custom_params, captured_output_store, disable_progress)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/pythonpath/openpilot/selfdrive/test/process_replay/process_replay.py", line 708, in _replay_multi_process
    container.start(params_config, env_config, all_msgs, frs, fingerprint, captured_output_store is not None)
  File "/data/pythonpath/openpilot/selfdrive/test/process_replay/process_replay.py", line 261, in start
    while not all(self.pm.all_readers_updated(s) for s in self.cfg.pubs if s not in self.cfg.ignore_alive_pubs):
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/pythonpath/openpilot/common/timeout.py", line 20, in handle_timeout
    raise TimeoutException(self.error_msg)
openpilot.common.timeout.TimeoutException: timed out waiting for process to start: 'modeld'
Stopping listener for: camerad
comma@comma-8fcff1ab:/data/openpilot/selfdrive/test/process_replay$

Provide a route where the issue occurs

N/A

openpilot version

master

Additional info

No response

adeebshihadeh commented 1 month ago

Likely need to disable power save mode: scripts/disable_powersave.py.