Possible Bug--Collision Sensor failed to report collision event in several scenarios

[AOOOOOA]

CARLA version: 0.9.13

Platform/OS: ubuntu 18.04

Problem you have experienced: Possible Bug: The collision sensor doesn't work in some conditions. What you expected to happen: The collision sensor should report the collision event whenever the collision happens.

Steps to reproduce: We used Scenario Runner and tested different scenarios like FollowLeadingVehicle, IntersectionCollisionAvoidance, LaneChangeSimple, etc. We discovered that sometimes collisions were visible to the naked eye, but the collision detector did not report the event. Here are some videos and the corresponding parameters:

https://github.com/carla-simulator/carla/assets/38175747/21aa04b3-c1a8-49a5-b3d6-0ccaa3cb26b1

Above is the FollowLeadingVehicle scenario, we set the initial driving speed of the vehicle is 20, which does not affect the final result because it seems that the vehicle doesn't need the acceleration process when changing speed in Carla. When the ego vehicle is 4m away from the NPC actor, we change the driving speed to 6, and change the driving angle to (x = 1 and y = -0.31).

initial speed: 20 distance to change driving behavior: 4m speed after change driving behavior: 6 driving angle after change driving behavior: (x = 1, y = -0.31)

The collision sensor attached on ego vehicle doesn't report the collision event. For the sake of caution and fairness, we also attached a collision sensor on the NPC actor, it also failed to report the incident. We have run multiple rounds of tests in different scenarios and this problem appeared under different parameters. We are working on finding some patterns in the data results. Now at first glance, it looks like low speed, pedestrians, and bicycles are a few of the elements more prone to this problem, however, there are also cases of non-report when NPC is a vehicle. We will report the conclusion later.

[lfxx]

CARLA version: 0.9.13

Platform/OS: ubuntu 18.04

Problem you have experienced: Possible Bug: The collision sensor doesn't work in some conditions. What you expected to happen: The collision sensor should report the collision event whenever the collision happens.

Steps to reproduce: We used Scenario Runner and tested different scenarios like FollowLeadingVehicle, IntersectionCollisionAvoidance, LaneChangeSimple, etc. We discovered that sometimes collisions were visible to the naked eye, but the collision detector did not report the event. Here are some videos and the corresponding parameters:

video.mp4 Above is the FollowLeadingVehicle scenario, we set the initial driving speed of the vehicle is 20, which does not affect the final result because it seems that the vehicle doesn't need the acceleration process when changing speed in Carla. When the ego vehicle is 4m away from the NPC actor, we change the driving speed to 6, and change the driving angle to (x = 1 and y = -0.31).

initial speed: 20 distance to change driving behavior: 4m speed after change driving behavior: 6 driving angle after change driving behavior: (x = 1, y = -0.31)

The collision sensor attached on ego vehicle doesn't report the collision event. For the sake of caution and fairness, we also attached a collision sensor on the NPC actor, it also failed to report the incident. We have run multiple rounds of tests in different scenarios and this problem appeared under different parameters. We are working on finding some patterns in the data results. Now at first glance, it looks like low speed, pedestrians, and bicycles are a few of the elements more prone to this problem, however, there are also cases of non-report when NPC is a vehicle. We will report the conclusion later.

Same issue here.Hope for fixing this issue.

[AOOOOOA]

We have indeed identified the issue of "collision but collision sensor failed to detect". We will soon post and update the list of it. Could you please verify the bugs? @bernatx @Axel1092 @corkyw10 @marcgpuig @nsubiron @germanros1987

[AOOOOOA]

We have identified 9 different types of events in which the collision sensor fails to work.

https://docs.google.com/spreadsheets/d/1gCQAl9F7e9Co85OTAlQ0VIifq6cCFWIvEWT9FE1vWzo/edit?usp=sharing

We provided the environment configurations and some control parameters in the sheet. Our test scenarios are selected from scenario runner. Could you please investigate these problems and provide an update or patch to address them？

Here are two sample videos for you to refer. Additional videos can be found in the sheet. If you need more information, please feel free to ask me.

https://github.com/carla-simulator/carla/assets/38175747/2baf4fe1-ad36-4a4d-a98c-24b8fafb04fa

https://github.com/carla-simulator/carla/assets/38175747/d4f01cd2-c149-4bbe-8f72-fc4b7f454d0f

[AOOOOOA]

Hi, I found some users reported issues with the collision sensor of the old version(ref https://github.com/carla-simulator/carla/issues/3183). It is said that the old issues were fixed in Carla0.9.10 but I found issues with the collision sensor still exist in Carla 0.9.13. It seems like the old issues haven't been totally fixed or the last fix brings new problems. Could you please check? Thanks. @Blyron @XGodina @MattRoweEAIF @MattRowe18 @bernatx

[Blyron]

Hello!

Could supply the code which you found the issue? Thanks!

[AOOOOOA]

Sure! Below is the code I use.

Usage:

The usage is:

1. Run the scenario runner code to launch a pre-defined scenario:

python scenario_runner.py --openscenario srunner/examples/FollowLeadingVehicle.xosc --record all_temp_data/temp_data0 --port=2000 (for example here I use the FollowLeadingVehicle.xosc, it may need to change to other related scenarios)

2. Then run the code snippet I provided below with the configuration data in exact configuration part, which is used for controlling the ego vehicle. For example:

python ego_vehicle_github.py FollowLeadingVehicle 20 1 0 5 8 0.15 -0.01 100 all_temp_data/temp_data0 result.txt 2000

(I named it as ego_vehicle_github.py, and the args corresponding to the initial driving speed, initial driving angle x, initial driving angle y, distance between ego vehicle and other actor to change the driving behavior, new driving speed, new driving angle x, new driving angle y, etc.
You can read the args part in the attached code for the meaning)

Exact configuration:

Here are some samples that will create the bug scenarios, if you run it, and it report a normal collision event or no collision happened, try to run it multiple times since the simulation has some randomness.

FollowLeadingVehicle:

1. python ego_vehicle_github.py FollowLeadingVehicle 20 1 0 5 12 0.15 -0.01 2000
2. python ego_vehicle_github.py FollowLeadingVehicle 20 1 0 6 34 0.8 -0.45 2000
3. python ego_vehicle_github.py FollowLeadingVehicle 20 1 0 6 14 0.9 -0.05 2000

PedestrianCrossingFront Change the scenario runner scenario at first: python scenario_runner.py --openscenario srunner/examples/PedestrianCrossingFront.xosc --record all_temp_data/temp_data0 --port=2000

1. python ego_vehicle_github.py PedestrianCrossingFront 20 -1 0 3 12 -0.13 0.05 2000
2. python ego_vehicle_github.py PedestrianCrossingFront 20 -1 0 5 40 -0.97 -0.95 2000

LaneChangeSimple: Change the scenario runner scenario at first: python scenario_runner.py --openscenario srunner/examples/LaneChangeSimple.xosc --record all_temp_data/temp_data0 --port=2000 then:

1. python ego_vehicle_github.py LaneChangeSimple 50 0 1 5 20 0.8 0 2000
2. python ego_vehicle_github.py LaneChangeSimple 40 0 1 7.1 14 0.5 0.811 2000

from __future__ import print_function

import argparse
import glob
import math
import os
import sys
import time
try:
    sys.path.append(glob.glob('../carla/dist/carla-*%d.%d-%s.egg' % (
        sys.version_info.major,
        sys.version_info.minor,
        'win-amd64' if os.name == 'nt' else 'linux-x86_64'))[0])
except IndexError:
    pass
try:
    sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + '/carla')
except IndexError:
    pass

import carla

#record the result as json to the same folder
def _on_collision(event):
    if event.other_actor.type_id != "static.road":
        if len(event.other_actor.attributes) != 0:
            print("COLLISION:", event)
            print("Collision Happen!")

def ego_vehicle(args):
    actor_list =  []
    player = None
    other_actor=None
    try:
        try:
            #create client
            client = carla.Client('localhost', args.port)
            client.set_timeout(2.0)
            #world connection
            world = client.get_world() 
            #get blueprint libarary
            blueprint_library = world.get_blueprint_library()

            snapshot0 = world.get_snapshot()
            first_frame_id = snapshot0.frame

            print("waiting for ego vehicle ....")
            possible_vehicles = world.get_actors().filter('vehicle.*')
            for vehicle in possible_vehicles:
                if vehicle.attributes['role_name'] == 'hero':
                    print("Ego vehicle found")
                    player = vehicle
                    continue
                if vehicle.attributes['role_name'] == 'adversary':
                    print("Other vehicle found")
                    other_actor=vehicle
                    continue
            if other_actor==None:
                possible_walkers=world.get_actors().filter('walker.*')
                for walker in possible_walkers:
                    if walker.attributes['role_name'] == 'adversary':
                        print("Other walker found")
                        other_actor=walker
                        continue

            if player is not None:
                spectator=world.get_spectator()
                actor_list.append(spectator)

                actor_list.append(player)

                collision_bp = blueprint_library.find('sensor.other.collision')
                sensor_collision = world.spawn_actor(collision_bp, carla.Transform(),attach_to=player)

                sensor_collision.listen(lambda event: _on_collision(event))
                actor_list.append(sensor_collision)

                while True:

                    transform=player.get_transform() 
                    spectator.set_transform(carla.Transform(transform.location + carla.Location(z=20),carla.Rotation(pitch=-90)))

                    snapshot=world.get_snapshot()
                    cur_frame_id=snapshot.frame
                    num_frames=cur_frame_id-first_frame_id

                    ego_location = player.get_location()
                    other_location = other_actor.get_location()

                    distance = math.sqrt((ego_location.x - other_location.x)**2 + 
                                        (ego_location.y - other_location.y)**2 + 
                                        (ego_location.z - other_location.z)**2)

                    transform=player.get_transform()

                    init_direction= carla.Vector3D(x=args.init_direction_x ,y=args.init_direction_y ,z=0)  #set the init direction, init speed, new_speed, new_direction as mutated parameters
                    player.set_target_velocity(init_direction*args.init_speed)

                    #new driving direction when the distance between ego vehicle and other vehicle is less than a threshold
                    new_direction=carla.Vector3D(x=args.d_x,y=args.d_y,z=0.0)                    
                    if distance < args.distance:
                        player.set_target_velocity(new_direction*args.speed) 

                    if num_frames >2500:
                        return actor_list,"end"

                    if args.sync:
                        world.tick()

                    else:
                        world.wait_for_tick()

            else:
                print("Cannot find the ego vehicle")

        finally:
            pass

    except KeyboardInterrupt:
        print('\nCancelled by user. Bye!')

def main():

    argparser = argparse.ArgumentParser(
        description='CARLA Ego Vehicle Controller')

    argparser.add_argument(
        '--sync',
        action='store_true',
        help='Synchronous mode execution')
    argparser.add_argument(
        '-m',"--manual",type=bool,
        default=False,
        help='manually allocate the running command')

    argparser.add_argument(
        "-a", "--agent", type=str,
        choices=["Behavior", "Basic","Maneuver"],
        help="select which agent to run",
        default="Basic")

    argparser.add_argument(
        '-b', '--behavior', type=str,
        choices=["cautious", "normal", "aggressive"],
        help='Choose one of the possible agent behaviors (default: normal) ',
        default='normal')

    argparser.add_argument(
        'scenario_name', type= str,
        help='scenario name in scenario runner',
        default='FollowLeadingVehicle')

    argparser.add_argument(
        'init_speed', type=float,
        help='init speed of the ego vehicle',
        default='5')

    argparser.add_argument(
        'init_direction_x', type=float,
        help='the initial direction x of the ego vehicle',
        default='1')

    argparser.add_argument(
        'init_direction_y', type=float,
        help='the initial direction y of the ego vehicle',
        default='0')

    argparser.add_argument(
        'distance', type= float,
        help='distance when the ego vehicle begin to turn',
        default='5')

    argparser.add_argument(
        'speed', type=float,
        help='the turn speed of the ego vehicle',
        default='5')

    argparser.add_argument(
        'd_x', type=float,
        help='the turn direction x of the ego vehicle',
        default='1')

    argparser.add_argument(
        'd_y', type=float,
        help='the turn direction y of the ego vehicle',
        default='0')

    argparser.add_argument(
        'port', type=int,
        default='2000',
        help='TCP port to listen to (default: 2000)')

    args=argparser.parse_args()

    actor_list, state=ego_vehicle(args)
    if state=="end":
        #kill the actor_list
        for actor in actor_list:
            actor.destroy()
        print("All cleaned up!")

    time.sleep(3)

if __name__ == '__main__':
    main()

[AOOOOOA]

Hello!

Could supply the code which you found the issue? Thanks!

Hi Blyron, I posted the related code and configuration, please have a try. I use Carla 0.9.13 on ubuntu18.04. If you run the code and the simulator reports a normal collision or non-collision, please try to run it multiple times since the simulation randomness. If you have any further questions, please let me know.

Thanks!

[Blyron]

Hello! We have detected the issue, it will be fixed soon.

[Blyron]

Hello!

Could you try this adding the attribute is invicible false to your walkers BP?

  # Spawn the actor
  bp = bp_lib.filter("*walker*")[0]
  bp.set_attribute('is_invincible', 'false')
  walker = world.spawn_actor(bp, carla.Transform(carla.Location(200.7, 199.9, 0.2), carla.Rotation()))

Vehicles are not colliding with invincible walkers

[AOOOOOA]

Hello!

Could you try this adding the attribute is invicible false to your walkers BP?

  # Spawn the actor
  bp = bp_lib.filter("*walker*")[0]
  bp.set_attribute('is_invincible', 'false')
  walker = world.spawn_actor(bp, carla.Transform(carla.Location(200.7, 199.9, 0.2), carla.Rotation()))

Vehicles are not colliding with invincible walkers

Hi Blyron, I will try and tell you the result. BTW, not only pedestrians have the error, I forgot to mention that I replaced the vehicle in FollowLeadingVehicel.xsoc as a bicycle which also belongs to the vehicle class in Carla. Thus we have the error video as shown in the above comments. I provided the details in https://github.com/carla-simulator/carla/issues/7394#issuecomment-2058130066 for you to refer.

Thanks.

[Blyron]

We are fixing pedestrians soon. At least for the moment for bycicles will take more time which we have not right now, but we will look into it in the near future

[AOOOOOA]

We are fixing pedestrians soon. At least for the moment for bycicles will take more time which we have not right now, but we will look into it in the near future

Glad to hear that! I will try the invincible pedestrian code in my project and give you the results. Could you please help to apply for a CVE ID for this vulnerability as the developer? Thanks.

[Blyron]

This should be fixed in the latest dev, branch

[Blyron]

I do not know how to do that. If you have any issues just link to this post

[AOOOOOA]

Hi Blyron,

Sorry for the late response. I tried the "is_invincible" option for pedestrians, but Carla still failed to detect it. I have tested the latest version of Carla 0.9.15 (after you fix the bug), and the bug seems to still exist. I also found some interesting findings; I compared the pre-release version of Carla 0.9.13 (in .tar.gz) and the compiled version; some ignored collision cases shown in the pre-release version but not shown in the compiled. I don't know why, but maybe you can dig into it.

I found two more new conditions in which the ignored collisions would happen.

This one can be run with: python ego_vehicle_github.py LaneChangeSimple 50 0 1 5 20 0.95 0.18 100

python ego_vehicle_github.py IntersectionCrossingAvoidance 10 -1 0 4 31 -0.35 -0.75 100

Maybe you can have a try and find out whether they share the same failure reason.