OpenDriveLab / OpenLane-V2

[NeurIPS 2023 Track Datasets and Benchmarks] OpenLane-V2: The First Perception and Reasoning Benchmark for Road Driving
https://proceedings.neurips.cc/paper_files/paper/2023/hash/3c0a4c8c236144f1b99b7e1531debe9c-Abstract-Datasets_and_Benchmarks.html
Apache License 2.0
541 stars 65 forks source link

baseline_large performance #22

Closed qwertyczx closed 1 year ago

qwertyczx commented 1 year ago

Hello, is there some information about the result of baseline_large in evaluation?

faikit commented 1 year ago

Hi, we have some modifications on augmentation etc., but the following results on the val split can be used for reference:

OpenLane-V2 Score - 0.292732
    DET_l - 0.183719
    DET_l_chamfer - 0.168729
    DET_t - 0.457165
    TOP_ll - 0.022970
    TOP_lt - 0.143250
F-Score - 0.215878
qwertyczx commented 1 year ago

OK. Thanks!

wudongming97 commented 1 year ago

Hi, we have some modifications on augmentation etc., but the following results on the val split can be used for reference:

OpenLane-V2 Score - 0.292732
    DET_l - 0.183719
    DET_l_chamfer - 0.168729
    DET_t - 0.457165
    TOP_ll - 0.022970
    TOP_lt - 0.143250
F-Score - 0.215878

Hi, what is the relationship between the baseline_large and TopoNet https://github.com/OpenDriveLab/TopoNet ? I can see they sometimes share same hyper-parameters.

faikit commented 1 year ago

Hi, they are unrelative.

YuanxianH commented 1 year ago

Hi, we have some modifications on augmentation etc., but the following results on the val split can be used for reference:

OpenLane-V2 Score - 0.292732
    DET_l - 0.183719
    DET_l_chamfer - 0.168729
    DET_t - 0.457165
    TOP_ll - 0.022970
    TOP_lt - 0.143250
F-Score - 0.215878

Will this version of the baseline be open-sourced in the future? If so, when? Thanks a lot.

faikit commented 1 year ago

Hi, this version is the baseline_large, the modification should not be huge. The reported scores might differ a bit since the change of code, data version, etc.

YuanxianH commented 1 year ago

Thank you for your reply. Would you mind sharing any additional data augmentation methods you have used compared to baseline_large? Something like random rotation or translation in BEV? 🤔 Besides, is there any reference paper or code that can be referenced? 😄

faikit commented 1 year ago

The addtional augmentations is not new, you can find them in like camera-based 3d detection models or something similar.