CollaborativePerception / V2Xverse

77 stars 10 forks source link

UnboundLocalError: local variable 'transform' referenced before assignment #23

Closed hollow-503 closed 4 weeks ago

hollow-503 commented 1 month ago

Thank you for your impressive work! I am generating dataset by myself with given command.

# Generate a dataset in parallel

cd V2Xverse
# Initialize dataset directory
python ./simulation/data_collection/init_dir.py --dataset_dir  ./dataset

# Generate scripts for every routes
python ./simulation/data_collection/generate_scripts.py

# Link dataset directory, if you initialized dataset in other directory, replace ./dataset with your dataset directory
ln -s ${PWD}/dataset/ ./external_paths/data_root

# Open Carla server (15 parallel process in total)
CUDA_VISIBLE_DEVICES=0 ./external_paths/carla_root/CarlaUE4.sh --world-port=40000 -prefer-nvidia
CUDA_VISIBLE_DEVICES=1 ./external_paths/carla_root/CarlaUE4.sh --world-port=40002 -prefer-nvidia
CUDA_VISIBLE_DEVICES=2 ./external_paths/carla_root/CarlaUE4.sh --world-port=40004 -prefer-nvidia
...
CUDA_VISIBLE_DEVICES=7 ./external_paths/carla_root/CarlaUE4.sh --world-port=40028 -prefer-nvidia

# Execute data generation in parallel
bash simulation/data_collection/generate_v2xverse_all.sh

However, there are errors when I generating some of scenarios:

...
Setup waypoint disturb!                                                                                                                        
destroy hazard actor: id: 5157, type: vehicle.audi.tt                                                                                          
destroy hazard actor: id: 5207, type: vehicle.lincoln.mkz2017                                                                                  
destroy hazard actor: id: 4598, type: vehicle.lincoln.mkz2017                                                                                  
destroy hazard actor: id: 4603, type: vehicle.lincoln.mkz2017                                                                                  
destroy hazard actor: id: 4508, type: vehicle.tesla.cybertruck                                                                                 
destroy hazard actor: id: 4594, type: vehicle.gazelle.omafiets                                                                                 
destroy hazard actor: id: 5116, type: vehicle.mini.cooperst                                                                                    
destroy hazard actor: id: 5136, type: vehicle.mercedes-benz.coupe                                                                              
destroy hazard actor: id: 5198, type: vehicle.gazelle.omafiets                                                                                 
destroy hazard actor: id: 4528, type: vehicle.volkswagen.t2                                                                                    
destroy hazard actor: id: 4516, type: vehicle.nissan.micra                                                                                     
destroy hazard actor: id: 4534, type: vehicle.kawasaki.ninja                                                                                   
destroy hazard actor: id: 4574, type: vehicle.seat.leon                                                                                        
destroy hazard actor: id: 4522, type: vehicle.bmw.grandtourer                                                                                  
destroy hazard actor: id: 5525, type: vehicle.diamondback.century                                                                              
destroy hazard actor: id: 5531, type: vehicle.diamondback.century                                                                              
destroy hazard actor: id: 5115, type: vehicle.mustang.mustang                                                                                  
destroy hazard actor: id: 5141, type: vehicle.mercedes-benz.coupe                                                                              
destroy hazard actor: id: 5182, type: vehicle.dodge_charger.police                                                                             
destroy hazard actors failed                                                                                                                   
Exception in thread Thread-2:
Traceback (most recent call last):                                                                                                             
  File "/home/xxx/anaconda3/envs/v2xverse/lib/python3.7/threading.py", line 926, in _bootstrap_inner                                   
    self.run()                                                                                                                                 
  File "/home/xxx/anaconda3/envs/v2xverse/lib/python3.7/threading.py", line 870, in run                                                
    self._target(*self._args, **self._kwargs)                                                                                                  
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/envs/sensor_interface.py", line 72, in run                                
    self._callback(GenericMeasurement(self.__call__(), GameTime.get_frame()))                                                                  
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/envs/sensor_interface.py", line 140, in __call__                          
    speed, vel_np, acc_np, vel_angle_np, transform_dict = self._get_forward_speed(transform=transform, velocity=velocity, acc=acc, vel_angle=ve
l_angle)
UnboundLocalError: local variable 'transform' referenced before assignment

destroy hazard actor: id: 5171, type: vehicle.toyota.prius

Error during the simulation:
> A sensor took too long to send their data

Traceback (most recent call last):
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/envs/sensor_interface.py", line 266, in get_data
    sensor_data = self._new_data_buffers.get(True, self._queue_timeout) 
  File "/xxx/anaconda3/envs/v2xverse/lib/python3.7/queue.py", line 178, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/scenarios/scenario_manager.py", line 188, in _tick_scenario
    ego_action = self._agent()
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/autoagents/agent_wrapper.py", line 110, in __call__
    return self._agent()
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/autoagents/autonomous_agent.py", line 110, in __call__
    input_data = self.sensor_interface.get_data() # gps, lidar, imu, rgb, speed
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/envs/sensor_interface.py", line 270, in get_data
    raise SensorReceivedNoData("A sensor took too long to send their data")
leaderboard.envs.sensor_interface.SensorReceivedNoData: A sensor took too long to send their data

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "simulation/leaderboard/leaderboard/leaderboard_evaluator_parameter.py", line 486, in _load_and_run_scenario
    self.manager.run_scenario()
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/scenarios/scenario_manager.py", line 171, in run_scenario
    self._tick_scenario(timestamp)
  File "/xxx/V2Xverse/simulation/leaderboard/leaderboard/scenarios/scenario_manager.py", line 192, in _tick_scenario
    raise RuntimeError(e)
RuntimeError: A sensor took too long to send their data

> Stopping the route

========= Results of RouteScenario_4 (repetition 0) ------ FAILURE =========

╒═════════════════════════════════╤═════════════════════╕
│ Start Time                      │ 2024-08-05 20:43:38 │
├─────────────────────────────────┼─────────────────────┤
│ End Time                        │ 2024-08-05 21:55:49 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time)          │ 4330.81s            │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time)            │ 107.7s              │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.025               │
╘═════════════════════════════════╧═════════════════════╛

╒═══════════════════════╤═════════╤═════════╕                                                                                                  
│ Criterion             │ Result  │ Value   │                                                                                                  
├───────────────────────┼─────────┼─────────┤                                                                                                  
│ RouteCompletionTest   │ FAILURE │ 21.44 % │                                                                                                  
├───────────────────────┼─────────┼─────────┤                                                                                                  
│ OutsideRouteLanesTest │ SUCCESS │ 0 %     │                                                                                                  
├───────────────────────┼─────────┼─────────┤                                                                                                  
│ CollisionTest         │ SUCCESS │ 0 times │                                                                                                  
├───────────────────────┼─────────┼─────────┤                                                                                                  
│ RunningRedLightTest   │ SUCCESS │ 0 times │                                                                                                  
├───────────────────────┼─────────┼─────────┤
│ RunningStopTest       │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ InRouteTest           │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ AgentBlockedTest      │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ Timeout               │ SUCCESS │         │
╘═══════════════════════╧═════════╧═════════╛

> Registering the route statistics                                                                                                             
> Registering the route statistics                                                                                                             
ERROR: failed to destroy actor 4508 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4511 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4516 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4522 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4524 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4528 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4534 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4539 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4563 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4574 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4588 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4594 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4598 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4603 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4785 : unable to destroy actor: not found                                                                       
ERROR: failed to destroy actor 4697 : unable to destroy actor: not found                                                                       
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5413 (sensor.lidar.ray_cast)                                 
ERROR: failed to destroy actor 4699 : unable to destroy actor: not found                                                                       
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5414 (sensor.camera.rgb)                                     
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5415 (sensor.camera.rgb)                                     
ERROR: failed to destroy actor WARNING: attempting to unsubscribe from stream but sensor wasn't listening:4701 : unable to destroy actor: not f
ound                                                                                                                                           
 Actor 5418 (sensor.lidar.ray_cast_semantic) 

WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5419 (sensor.camera.semantic_segmentation)                   
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5420 (sensor.camera.depth) 
ERROR: failed to destroy actor 4703 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5421 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor 4705 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5422 (sensor.camera.depth) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5423 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5424 (sensor.camera.depth) 
4707 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5425 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor 4709 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5426 (sensor.camera.depth) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5427 (sensor.lidar.ray_cast) 
ERROR: failed to destroy actor 4711 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5428 (sensor.camera.rgb) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5429 (sensor.camera.rgb) 
ERROR: failed to destroy actorWARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5430 (sensor.camera.rgb) 
 4713 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4715 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5431 (sensor.camera.rgb) 
ERROR: failed to destroy actor 4717 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5432 (sensor.lidar.ray_cast_semantic) 
ERROR: failed to destroy actor WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5433 (sensor.camera.semantic_s
egmentation) 
4719 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5434 (sensor.camera.depth) 
ERROR: failed to destroy actor 4721 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5435 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor 4723 : unable to destroy actor: not found

ERROR: failed to destroy actor 4725 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5437 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor 4727 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5438 (sensor.camera.depth) 
ERROR: failed to destroy actor 4729 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5439 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor 4731 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5440 (sensor.camera.depth) 
ERROR: failed to destroy actor 4733 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5416 (sensor.camera.rgb) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 4788 (sensor.other.collision) 
ERROR: failed to destroy actor 4735 : unable to destroy actor: not found 
ERROR: failed to destroy actor WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 5417 (sensor.camera.rgb) 4737 
: 
unable to destroy actor: not found  
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 4789 (sensor.other.collision) 
ERROR: failed to destroy actor 4739 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4741 : unable to destroy actor: not found 
> Registering the global statistics 
ERROR: failed to destroy actor 4743 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4745 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4747 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4749 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4751 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4753 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4755 : unable to destroy actor: not found
ERROR: failed to destroy actor 4757 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4759 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4761 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4763 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4765 : unable to destroy actor: not found 
ERROR: failed to destroy actor 4767 : unable to destroy actor: not found
hollow-503 commented 1 month ago

And it also reports Segmentation fault:

> Running the route                                                                                                                            
Setup waypoint disturb!                                                                                                                        
destroy hazard actor: id: 5161, type: vehicle.chevrolet.impala                                                                                 
destroy hazard actor: id: 5127, type: vehicle.tesla.model3                                                                                     
destroy hazard actor: id: 5140, type: vehicle.lincoln.mkz2017                                                                                  
destroy hazard actor: id: 5156, type: vehicle.jeep.wrangler_rubicon                                                                            
destroy hazard actor: id: 5196, type: vehicle.carlamotors.carlacola                                                                            
destroy hazard actor: id: 5114, type: vehicle.tesla.cybertruck                                                                                 
destroy hazard actor: id: 5117, type: vehicle.mustang.mustang                                                                                  
destroy hazard actor: id: 5167, type: vehicle.gazelle.omafiets                                                                                 
destroy hazard actor: id: 5121, type: vehicle.mustang.mustang                                                                                  
destroy hazard actor: id: 5492, type: vehicle.diamondback.century                                                                              
./simulation/data_collection/scripts/weather-0/routes_town01_20.sh: line 50: 3067665 Segmentation fault      python ${LEADERBOARD_ROOT}/leaderb
oard/leaderboard_evaluator_parameter.py --scenarios=${SCENARIOS} --scenario_parameter=${SCENARIOS_PARAMETER} --routes=${ROUTES} --repetitions=$
{REPETITIONS} --track=${CHALLENGE_TRACK_CODENAME} --checkpoint=${CHECKPOINT_ENDPOINT} --agent=${TEAM_AGENT} --agent-config=${TEAM_CONFIG} --deb
ug=${DEBUG_CHALLENGE} --record=${RECORD_PATH} --resume=${RESUME} --port=${PORT} --trafficManagerPort=${TM_PORT} --carlaProviderSeed=${CARLA_SEE
D} --trafficManagerSeed=${TRAFFIC_SEED} --ego-num=${EGO_NUM} --timeout 600 

Carla reports following errors at the same time:

ERROR: Invalid session: no stream available with id 125
...
ERROR: Invalid session: no stream available with id 125 
ERROR: session 5932 : error retrieving stream id : End of file

Could you kindly help me or give me some advices? Thank you very much! Looking forward to hearing from you.

gjliu9 commented 1 month ago

And it also reports Segmentation fault:

> Running the route                                                                                                                            
Setup waypoint disturb!                                                                                                                        
destroy hazard actor: id: 5161, type: vehicle.chevrolet.impala                                                                                 
destroy hazard actor: id: 5127, type: vehicle.tesla.model3                                                                                     
destroy hazard actor: id: 5140, type: vehicle.lincoln.mkz2017                                                                                  
destroy hazard actor: id: 5156, type: vehicle.jeep.wrangler_rubicon                                                                            
destroy hazard actor: id: 5196, type: vehicle.carlamotors.carlacola                                                                            
destroy hazard actor: id: 5114, type: vehicle.tesla.cybertruck                                                                                 
destroy hazard actor: id: 5117, type: vehicle.mustang.mustang                                                                                  
destroy hazard actor: id: 5167, type: vehicle.gazelle.omafiets                                                                                 
destroy hazard actor: id: 5121, type: vehicle.mustang.mustang                                                                                  
destroy hazard actor: id: 5492, type: vehicle.diamondback.century                                                                              
./simulation/data_collection/scripts/weather-0/routes_town01_20.sh: line 50: 3067665 Segmentation fault      python ${LEADERBOARD_ROOT}/leaderb
oard/leaderboard_evaluator_parameter.py --scenarios=${SCENARIOS} --scenario_parameter=${SCENARIOS_PARAMETER} --routes=${ROUTES} --repetitions=$
{REPETITIONS} --track=${CHALLENGE_TRACK_CODENAME} --checkpoint=${CHECKPOINT_ENDPOINT} --agent=${TEAM_AGENT} --agent-config=${TEAM_CONFIG} --deb
ug=${DEBUG_CHALLENGE} --record=${RECORD_PATH} --resume=${RESUME} --port=${PORT} --trafficManagerPort=${TM_PORT} --carlaProviderSeed=${CARLA_SEE
D} --trafficManagerSeed=${TRAFFIC_SEED} --ego-num=${EGO_NUM} --timeout 600 

Carla reports following errors at the same time:

ERROR: Invalid session: no stream available with id 125
...
ERROR: Invalid session: no stream available with id 125 
ERROR: session 5932 : error retrieving stream id : End of file

Could you kindly help me or give me some advices? Thank you very much! Looking forward to hearing from you.

Hi, thanks for your attention. The error "A sensor took too long to send their data" and "Segmentation fault" are most likely caused by the busy system processes. When you are running on a busy server with many other processes, these two errors may occur. Running on a less busy server or reducing the number of parallel processes may get you on the way.

hollow-503 commented 1 month ago

Thank you for your reply! Although there are no segmentation fault or errors are reported, the platform sometimes generates 1 or 2 FAILURE like this:

========= Results of RouteScenario_8 (repetition 0) ------ SUCCESS=========
╒═════════════════════════════════╤═════════════════════╕
│ Start Time                      │ 2024-08-07 14:18:46 │
├─────────────────────────────────┼─────────────────────┤
│ End Time                        │ 2024-08-07 15:07:06 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time)          │ 2899.98s            │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time)            │ 90.35s              │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.031               │
╘═════════════════════════════════╧═════════════════════╛

╒═══════════════════════╤═════════╤═════════╕
│ Criterion             │ Result  │ Value   │
├───────────────────────┼─────────┼─────────┤
│ RouteCompletionTest   │ SUCCESS │ 100 %   │
├───────────────────────┼─────────┼─────────┤
│ OutsideRouteLanesTest │ SUCCESS │ 0 %     │
├───────────────────────┼─────────┼─────────┤
│ CollisionTest         │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningRedLightTest   │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningStopTest       │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ InRouteTest           │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ AgentBlockedTest      │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ Timeout               │ SUCCESS │         │
╘═══════════════════════╧═════════╧═════════╛

2024-08-07 15:07:06  
========= Results of RouteScenario_8 (repetition 0) ------ FAILURE=========

╒═════════════════════════════════╤═════════════════════╕
│ Start Time                      │ 2024-08-07 14:18:46 │
├─────────────────────────────────┼─────────────────────┤
│ End Time                        │ 2024-08-07 15:07:06 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time)          │ 2899.98s            │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time)            │ 90.35s              │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.031               │
╘═════════════════════════════════╧═════════════════════╛

╒═══════════════════════╤═════════╤═════════╕
│ Criterion             │ Result  │ Value   │
├───────────────────────┼─────────┼─────────┤
│ RouteCompletionTest   │ FAILURE │ 37.36 % │
├───────────────────────┼─────────┼─────────┤
│ OutsideRouteLanesTest │ SUCCESS │ 0 %     │
├───────────────────────┼─────────┼─────────┤
│ CollisionTest         │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningRedLightTest   │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningStopTest       │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ InRouteTest           │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ AgentBlockedTest      │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ Timeout               │ SUCCESS │         │
╘═══════════════════════╧═════════╧═════════╛

Is that means I should regenerate the corresponding data? (or actually it was generated correctly?)

gjliu9 commented 1 month ago

Thank you for your reply! Although there are no segmentation fault or errors are reported, the platform sometimes generates 1 or 2 FAILURE like this:

========= Results of RouteScenario_8 (repetition 0) ------ SUCCESS=========
╒═════════════════════════════════╤═════════════════════╕
│ Start Time                      │ 2024-08-07 14:18:46 │
├─────────────────────────────────┼─────────────────────┤
│ End Time                        │ 2024-08-07 15:07:06 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time)          │ 2899.98s            │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time)            │ 90.35s              │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.031               │
╘═════════════════════════════════╧═════════════════════╛

╒═══════════════════════╤═════════╤═════════╕
│ Criterion             │ Result  │ Value   │
├───────────────────────┼─────────┼─────────┤
│ RouteCompletionTest   │ SUCCESS │ 100 %   │
├───────────────────────┼─────────┼─────────┤
│ OutsideRouteLanesTest │ SUCCESS │ 0 %     │
├───────────────────────┼─────────┼─────────┤
│ CollisionTest         │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningRedLightTest   │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningStopTest       │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ InRouteTest           │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ AgentBlockedTest      │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ Timeout               │ SUCCESS │         │
╘═══════════════════════╧═════════╧═════════╛

2024-08-07 15:07:06  
========= Results of RouteScenario_8 (repetition 0) ------ FAILURE=========

╒═════════════════════════════════╤═════════════════════╕
│ Start Time                      │ 2024-08-07 14:18:46 │
├─────────────────────────────────┼─────────────────────┤
│ End Time                        │ 2024-08-07 15:07:06 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time)          │ 2899.98s            │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time)            │ 90.35s              │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.031               │
╘═════════════════════════════════╧═════════════════════╛

╒═══════════════════════╤═════════╤═════════╕
│ Criterion             │ Result  │ Value   │
├───────────────────────┼─────────┼─────────┤
│ RouteCompletionTest   │ FAILURE │ 37.36 % │
├───────────────────────┼─────────┼─────────┤
│ OutsideRouteLanesTest │ SUCCESS │ 0 %     │
├───────────────────────┼─────────┼─────────┤
│ CollisionTest         │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningRedLightTest   │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningStopTest       │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ InRouteTest           │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ AgentBlockedTest      │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ Timeout               │ SUCCESS │         │
╘═══════════════════════╧═════════╧═════════╛

Is that means I should regenerate the corresponding data? (or actually it was generated correctly?)

The "Failure" means that the expert agent didn't finish the route, it might be blocked or met a timeout issue. But the driving data is collected anyway, even if this route is not completed, it can also serve as driving supervision.