Mistake Spotted in Paper's Scenario Collison Rate (SCR) Calculation Code

[SS47816]

Dear authors,

Very Impressive work!

However, I spotted a major issue in your calculation of the SCR.

Here is a snapshot of your current code (related Issue #11, commit: dfebc2c), which shows that instead of checking for collisions across different vehicles at the same timestep, you are checking for collisions on the same vehicle across different timesteps?

def get_SCR(pred):
    collide_idx = np.zeros(pred.shape[0])
    for i in range(pred.shape[0]):
        agent = WaymoAgent(pred[[i]])
        polygons = agent.get_polygon()
        for idx,poly in enumerate(polygons):
            for ano_idx,another_poly in enumerate(polygons):
                if poly.intersects(another_poly) and idx != ano_idx:
                    collide_idx[idx]=1

    return np.mean(collide_idx)

I am trying to reproduce the results in the paper. Could you help check this part and correct me if I am wrong?

Also, it was noticed that in your original paper, the collision rate is dependent on a "predefined IOU threshold", which was not reflected in your code using the shapely.Polygon APIs. May I ask what is the IOU threshold used to reproduce the SCR results in the paper?

Thank you!

[pengzhenghao]

@Alan-LanFeng

[Alan-LanFeng]

The 'WaymoAgent(pred[[i]])' creates an instance containing all the vehicles in a time step, which means that polygons are a list of all polygons of all vehicles. So the code is correct here. We simply use 0 as the IOU threshold in the paper and experiments.