search

OpenDriveLab / OpenLane-V2

[NeurIPS 2023 Track Datasets and Benchmarks] OpenLane-V2: The First Perception and Reasoning Benchmark for Road Driving
https://proceedings.neurips.cc/paper_files/paper/2023/hash/3c0a4c8c236144f1b99b7e1531debe9c-Abstract-Datasets_and_Benchmarks.html
Apache License 2.0
557 stars 66 forks source link

Evaluation on Eval.ai #29

Closed yuanpengtu closed 1 year ago

yuanpengtu commented 1 year ago

It seems that the evaluation is not started and keeps 'submitted' after submitting the file. And the status of another file that was submitted last Sunday keeps 'running' as well. Is there something wrong with the website? image

yuanpengtu commented 1 year ago

I think it should be fast to just compare the ground truth labels and prediction results but it actually costs plenty of time. So I doubt if there is something wrong with the website?

faikit commented 1 year ago

Hi, we just notice the problem and will fix it asap.

BTW, what about your evaluation on the val spilt on your local machine?

yuanpengtu commented 1 year ago

Hi, it seems that the website works now. Is the bug fixed now? As for the evaluation, I think it may not be good to list the results here : )

faikit commented 1 year ago

Hi, the evaluation error is caused by the limited computation resource, and we put more strict limitations on the uploaded files. What do you mean by here for listing the results?

yuanpengtu commented 1 year ago

I guess I may have misunderstood you : ) As the problem is solved, I will close this issue. Thanks for the timely help.