Parsing longest6 results Error

[MCUBE-2023]

Dear author, Sorry if asked many questions. But your work is very interesting, to an extent that sparked the interest of the community.

I got an error that I didn't know how to resolve related to Parsing longest6 results, particularly related to this instruction: ${WORK_DIR}/tools/result_parser.py --xml ${WORK_DIR}/leaderboard/data/longest6/longest6.xml --results /path/to/folder/with/json_results/ --save_dir /path/to/output --town_maps ${WORK_DIR}/leaderboard/data/town_maps_xodr.

So what I did, instead of defining the path directly as an argument in the instruction above itself, I modified the argument paths inside the file result_parser.py for each of --xml, --results, --save_dir, --town_maps, (please refer to the first screenshot attached to this issue).

Then after pointing to transfuser/tools, I run: python3 result_parser.py, but got this error: Traceback (most recent call last): File "result_parser.py", line 377, in main() File "result_parser.py", line 149, in main for infraction_name in evaluation_data['_checkpoint']['global_record']['infractions']: KeyError: 'infractions'

Referring to the screenshot attached, would you please tell me if I have adapted correctly the paths for the arguments? If yes, how can I resolve the issue related to encountered error, please ?

[MCUBE-2023]

I forget to attach the content of the file "transfuser_longest6.json" which was generated in the folder work_dir/transfuser/results after evaluating an agent with the script: ./leaderboard/scripts/local_evaluation.sh <working directory of this repo (*/transfuser/)>. I am copying here the content of "transfuser_longest6.json" because I think that maybe the content of this file have a certain correlation with the KeyError: 'infractions' generated when I run result_parser.py, hoping that you can provide me with insights on how to resolve this error of KeyError: 'infractions'. As you can notice in the content of "transfuser_longest6.json", there are no values in "infractions" cluster, contrarily to what I see in the file "autopilot_longest6.json" which I presume that this might be abnormal.

Here's the content of "transfuser_longest6.json":

{ "_checkpoint": { "global_record": {}, "progress": [], "records": [ { "index": 0, "infractions": { "collisions_layout": [], "collisions_pedestrian": [], "collisions_vehicle": [], "outside_route_lanes": [], "red_light": [], "route_dev": [], "route_timeout": [], "stop_infraction": [], "vehicle_blocked": [] }, "meta": { "duration_game": 407.65000607445836, "duration_system": 7629.184344530106, "route_length": 1130.24134093902 }, "route_id": "RouteScenario_0", "scores": { "score_composed": 70.21711038088651, "score_penalty": 1.0, "score_route": 70.21711038088651 }, "status": "Failed - Simulation crashed" } ] }, "eligible": false, "entry_status": "Crashed", "labels": [], "sensors": [ "carla_camera", "carla_camera", "carla_camera", "carla_imu", "carla_gnss", "carla_speedometer", "carla_lidar" ], "values": [] }

How Can I resolve this issue please ?

[Kait0]

You need to rerun the evaluation. The problem is that the transfuser_longest6.json file contains crashed routes: The message: "status": "Failed - Simulation crashed" indicates that the evaluation crashed midway through in RouteScenario_0 which means the resulting score is invalid.