CollaborativePerception / V2Xverse

89 stars 10 forks source link

Stopping the route, the agent has crashed: #29

Open fengxueguiren opened 1 month ago

fengxueguiren commented 1 month ago

When I generate data on one single route, the question about "Stopping the route, the agent has crashed:" arise.

transform:Transform(Location(x=338.770386, y=214.451157, z=0.500000), Rotation(pitch=360.000000, yaw=270.034607, roll=0.000000))
route_scenarios: {'Scenario10': 57}
set ip sensor for ego vehicle 0
set ip sensor for ego vehicle 1
> Running the route
Setup waypoint disturb!

Stopping the route, the agent has crashed:
> 

Traceback (most recent call last):
  File "/data/username/data/V2Xverse/simulation/leaderboard/leaderboard/scenarios/scenario_manager.py", line 188, in _tick_scenario
    ego_action = self._agent()
  File "/data/username/data/V2Xverse/simulation/leaderboard/leaderboard/autoagents/agent_wrapper.py", line 110, in __call__
    return self._agent()
  File "/data/username/data/V2Xverse/simulation/leaderboard/leaderboard/autoagents/autonomous_agent.py", line 118, in __call__
    control = self.run_step(input_data, timestamp)
  File "simulation/leaderboard/team_code/auto_pilot.py", line 416, in run_step
    mode='all'
  File "/data/username/data/V2Xverse/simulation/leaderboard/team_code/base_agent.py", line 665, in save
    self.actors_data, _ = process_lidar_visibility(self.actors_data, tick_data["lidar" + "_{}".format(self.vehicle_num)], lidar_pose, change_actor_file=True)
  File "simulation/leaderboard/team_code/eval_utils.py", line 245, in process_lidar_visibility
    num_lidar_points = get_points_in_rotated_box_3d(full_lidar, corners[N])
  File "simulation/leaderboard/team_code/eval_utils.py", line 106, in get_points_in_rotated_box_3d
    l1 = get_projection_length_for_vector_projection(p_rel, edge1)
  File "simulation/leaderboard/team_code/eval_utils.py", line 142, in get_projection_length_for_vector_projection
    assert np.sum(b ** 2, axis=-1) > 1e-6
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "simulation/leaderboard/leaderboard/leaderboard_evaluator_parameter.py", line 486, in _load_and_run_scenario
    self.manager.run_scenario()
  File "/data/username/data/V2Xverse/simulation/leaderboard/leaderboard/scenarios/scenario_manager.py", line 171, in run_scenario
    self._tick_scenario(timestamp)
  File "/data/username/data/V2Xverse/simulation/leaderboard/leaderboard/scenarios/scenario_manager.py", line 195, in _tick_scenario
    raise AgentError(e)
leaderboard.autoagents.agent_wrapper.AgentError
> Stopping the route

========= Results of RouteScenario_0 (repetition 0) ------ FAILURE =========

╒═════════════════════════════════╤═════════════════════╕
│ Start Time                      │ 2024-09-11 22:35:20 │
├─────────────────────────────────┼─────────────────────┤
│ End Time                        │ 2024-09-11 22:35:21 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time)          │ 0.69s               │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time)            │ 0.05s               │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.073               │
╘═════════════════════════════════╧═════════════════════╛

╒═══════════════════════╤═════════╤═════════╕
│ Criterion             │ Result  │ Value   │
├───────────────────────┼─────────┼─────────┤
│ RouteCompletionTest   │ FAILURE │ 0.0 %   │
├───────────────────────┼─────────┼─────────┤
│ OutsideRouteLanesTest │ SUCCESS │ 0 %     │
├───────────────────────┼─────────┼─────────┤
│ CollisionTest         │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningRedLightTest   │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningStopTest       │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ InRouteTest           │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ AgentBlockedTest      │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ Timeout               │ SUCCESS │         │
╘═══════════════════════╧═════════╧═════════╛

========= Results of RouteScenario_0 (repetition 0) ------ FAILURE =========

╒═════════════════════════════════╤═════════════════════╕
│ Start Time                      │ 2024-09-11 22:35:20 │
├─────────────────────────────────┼─────────────────────┤
│ End Time                        │ 2024-09-11 22:35:21 │
├─────────────────────────────────┼─────────────────────┤
│ Duration (System Time)          │ 0.69s               │
├─────────────────────────────────┼─────────────────────┤
│ Duration (Game Time)            │ 0.05s               │
├─────────────────────────────────┼─────────────────────┤
│ Ratio (System Time / Game Time) │ 0.073               │
╘═════════════════════════════════╧═════════════════════╛

╒═══════════════════════╤═════════╤═════════╕
│ Criterion             │ Result  │ Value   │
├───────────────────────┼─────────┼─────────┤
│ RouteCompletionTest   │ FAILURE │ 0.0 %   │
├───────────────────────┼─────────┼─────────┤
│ OutsideRouteLanesTest │ SUCCESS │ 0 %     │
├───────────────────────┼─────────┼─────────┤
│ CollisionTest         │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningRedLightTest   │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ RunningStopTest       │ SUCCESS │ 0 times │
├───────────────────────┼─────────┼─────────┤
│ InRouteTest           │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ AgentBlockedTest      │ SUCCESS │         │
├───────────────────────┼─────────┼─────────┤
│ Timeout               │ SUCCESS │         │
╘═══════════════════════╧═════════╧═════════╛

> Registering the route statistics
> Registering the route statistics
ERROR: failed to destroy actor 1623 : unable to destroy actor: not found 
ERROR: failed to destroy actor 1625 : unable to destroy actor: not found 
ERROR: failed to destroy actor 1627 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1761 (sensor.other.collision) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1762 (sensor.other.collision) 
ERROR: failed to destroy actor 1629 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1798 (sensor.lidar.ray_cast) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1799 (sensor.camera.rgb) 
ERROR: failed to destroy actor 1631 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1800 (sensor.camera.rgb) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1801 (sensor.camera.rgb) 
ERROR: failed to destroy actor 1633 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1802 (sensor.camera.rgb) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1803 (sensor.lidar.ray_cast_semantic) 
ERROR: failed to destroy actor 1635 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1804 (sensor.camera.semantic_segmentation) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1805 (sensor.camera.depth) 
ERROR: failed to destroy actor 1637 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1806 (sensor.camera.semantic_segmentation) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1807 (sensor.camera.depth) 
ERROR: failed to destroy actor 1639 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1808 (sensor.camera.semantic_segmentation) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1809 (sensor.camera.depth) 
ERROR: failed to destroy actor 1641 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1810 (sensor.camera.semantic_segmentation) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1811 (sensor.camera.depth) 
ERROR: failed to destroy actor 1643 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1812 (sensor.lidar.ray_cast) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1813 (sensor.camera.rgb) 
ERROR: failed to destroy actor 1645 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1814 (sensor.camera.rgb) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1815 (sensor.camera.rgb) 
ERROR: failed to destroy actor 1647 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1816 (sensor.camera.rgb) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1817 (sensor.lidar.ray_cast_semantic) 
ERROR: failed to destroy actor 1649 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1818 (sensor.camera.semantic_segmentation) 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1819 (sensor.camera.depth) 
ERROR: failed to destroy actor 1651 : unable to destroy actor: not found 
ERROR: failed to destroy actor 1653WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1820 (sensor.camera.semantic_segmentation) 
 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1821 (sensor.camera.depth) 
ERROR: failed to destroy actor 1655 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1822 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor 1657 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1823 (sensor.camera.depth) 
ERROR: failed to destroy actor 1659 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1824 (sensor.camera.semantic_segmentation) 
ERROR: failed to destroy actor 1661 : unable to destroy actor: not found 
WARNING: attempting to unsubscribe from stream but sensor wasn't listening: Actor 1825 (sensor.camera.depth) 
ERROR: failed to destroy actor 1663 : unable to destroy actor: not found 
> Registering the global statistics
ERROR: failed to destroy actor 1665 : unable to destroy actor: not found 
ERROR: failed to destroy actor 1667 : unable to destroy actor: not found 
fengxueguiren commented 1 month ago

I checked my dependencies. It might be that I am not using the recommended version of CARLA. I am in the process of trying.

fengxueguiren commented 1 month ago

I successfully ran it after replacing CARLA. However, on the 3090, the vehicles move once per second or more. And after a period of time, the following error occurs.

load Scenario10 at (x=87.55, y=228.6)
load Scenario10 at (x=58.44, y=325.72)
ego id:0
transform:Transform(Location(x=338.763153, y=226.451157, z=0.500000), Rotation(pitch=360.000000, yaw=270.034607, roll=0.000000))
ego id:1
transform:Transform(Location(x=338.770386, y=214.451157, z=0.500000), Rotation(pitch=360.000000, yaw=270.034607, roll=0.000000))
route_scenarios: {'Scenario10': 57}
set ip sensor for ego vehicle 0
set ip sensor for ego vehicle 1
> Running the route
Setup waypoint disturb!
./simulation/data_collection/scripts/weather-0/routes_town01_0.sh: 第 50 行: 732478 段错误               (核心已转储) python ${LEADERBOARD_ROOT}/leaderboard/leaderboard_evaluator_parameter.py --scenarios=${SCENARIOS} --scenario_parameter=${SCENARIOS_PARAMETER} --routes=${ROUTES} --repetitions=${REPETITIONS} --track=${CHALLENGE_TRACK_CODENAME} --checkpoint=${CHECKPOINT_ENDPOINT} --agent=${TEAM_AGENT} --agent-config=${TEAM_CONFIG} --debug=${DEBUG_CHALLENGE} --record=${RECORD_PATH} --resume=${RESUME} --port=${PORT} --trafficManagerPort=${TM_PORT} --carlaProviderSeed=${CARLA_SEED} --trafficManagerSeed=${TRAFFIC_SEED} --ego-num=${EGO_NUM} --timeout 6000
TianYifan-Vincent commented 1 month ago

I successfully ran it after replacing CARLA. However, on the 3090, the vehicles move once per second or more. And after a period of time, the following error occurs.

load Scenario10 at (x=87.55, y=228.6)
load Scenario10 at (x=58.44, y=325.72)
ego id:0
transform:Transform(Location(x=338.763153, y=226.451157, z=0.500000), Rotation(pitch=360.000000, yaw=270.034607, roll=0.000000))
ego id:1
transform:Transform(Location(x=338.770386, y=214.451157, z=0.500000), Rotation(pitch=360.000000, yaw=270.034607, roll=0.000000))
route_scenarios: {'Scenario10': 57}
set ip sensor for ego vehicle 0
set ip sensor for ego vehicle 1
> Running the route
Setup waypoint disturb!
./simulation/data_collection/scripts/weather-0/routes_town01_0.sh: 第 50 行: 732478 段错误               (核心已转储) python ${LEADERBOARD_ROOT}/leaderboard/leaderboard_evaluator_parameter.py --scenarios=${SCENARIOS} --scenario_parameter=${SCENARIOS_PARAMETER} --routes=${ROUTES} --repetitions=${REPETITIONS} --track=${CHALLENGE_TRACK_CODENAME} --checkpoint=${CHECKPOINT_ENDPOINT} --agent=${TEAM_AGENT} --agent-config=${TEAM_CONFIG} --debug=${DEBUG_CHALLENGE} --record=${RECORD_PATH} --resume=${RESUME} --port=${PORT} --trafficManagerPort=${TM_PORT} --carlaProviderSeed=${CARLA_SEED} --trafficManagerSeed=${TRAFFIC_SEED} --ego-num=${EGO_NUM} --timeout 6000

I met the same problem.

Jenny12388 commented 1 month ago

Anyone solve this issue?

Jenny12388 commented 1 month ago

I successfully ran it after replacing CARLA. However, on the 3090, the vehicles move once per second or more. And after a period of time, the following error occurs.

load Scenario10 at (x=87.55, y=228.6)
load Scenario10 at (x=58.44, y=325.72)
ego id:0
transform:Transform(Location(x=338.763153, y=226.451157, z=0.500000), Rotation(pitch=360.000000, yaw=270.034607, roll=0.000000))
ego id:1
transform:Transform(Location(x=338.770386, y=214.451157, z=0.500000), Rotation(pitch=360.000000, yaw=270.034607, roll=0.000000))
route_scenarios: {'Scenario10': 57}
set ip sensor for ego vehicle 0
set ip sensor for ego vehicle 1
> Running the route
Setup waypoint disturb!
./simulation/data_collection/scripts/weather-0/routes_town01_0.sh: 第 50 行: 732478 段错误               (核心已转储) python ${LEADERBOARD_ROOT}/leaderboard/leaderboard_evaluator_parameter.py --scenarios=${SCENARIOS} --scenario_parameter=${SCENARIOS_PARAMETER} --routes=${ROUTES} --repetitions=${REPETITIONS} --track=${CHALLENGE_TRACK_CODENAME} --checkpoint=${CHECKPOINT_ENDPOINT} --agent=${TEAM_AGENT} --agent-config=${TEAM_CONFIG} --debug=${DEBUG_CHALLENGE} --record=${RECORD_PATH} --resume=${RESUME} --port=${PORT} --trafficManagerPort=${TM_PORT} --carlaProviderSeed=${CARLA_SEED} --trafficManagerSeed=${TRAFFIC_SEED} --ego-num=${EGO_NUM} --timeout 6000

I met the same problem.

Did you end up solving it?