search

swc-17 / SparseDrive

SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation
MIT License
288 stars 31 forks source link

EVAL METRIC PROTOCAL #10

Closed turboxin closed 1 month ago

turboxin commented 1 month ago

It seems that UniAD and VAD used different eval metric protocal as noted by UAD and PARA-Drive paper

https://openaccess.thecvf.com/content/CVPR2024/papers/Weng_PARA-Drive_Parallelized_Architecture_for_Real-time_Autonomous_Driving_CVPR_2024_paper.pdf

https://arxiv.org/html/2406.17680v1

I'm wondering what did you used for comparision?

swc-17 commented 1 month ago

As described in paper, the L2 distance evaluation is aligned with VAD, and we re-implemented the evaluation protocol for collision rate.

zhiyozhao commented 1 month ago

The code shows that, the original VAD uses L2 metric averaged overtime, while sparsedrive use the current timestamps L2 metric:

VAD: image

Sparsedrive: image

swc-17 commented 1 month ago

Please refer to this issue.