search

ozendelait / rvc_devkit

Robust Vision Challenge Devkits
http://www.robustvision.net/
MIT License
107 stars 13 forks source link

Update Wilddash2 Submission (Semantic) #63

Closed gepu0221 closed 2 years ago

gepu0221 commented 2 years ago

Dear organizer, I aleady submited my results used the method name with postfix_RVC on the Wilddash2 dataset Now I want to update my submission with the same method name. However, it seems that the submission result cannot be updated. Moreover, I can't create new submission with the same method name. I want to how can I update my submission results. Can you give us some other advice? Thanks!

ozendelait commented 2 years ago

Hi gepu0221, as discussed on multiple occasions: the benchmarks are the final step after you are finished. There is a lot of public GT to test/finetune your solution and after you are satisfied you submit to the benchmark. It is not meant as a repeated debugging/optimization tool. If you changed your network, then you should need to "update" your submission to all other benchmarks as well?

gepu0221 commented 2 years ago

Hi gepu0221, as discussed on multiple occasions: the benchmarks are the final step after you are finished. There is a lot of public GT to test/finetune your solution and after you are satisfied you submit to the benchmark. It is not meant as a repeated debugging/optimization tool. If you changed your network, then you should need to "update" your submission to all other benchmarks as well?

Yes, we also need to update our submissions to other benchmarks. And other datasets can use the same method name to update the submission in limited times.

ozendelait commented 2 years ago

Ok, please finalize your solution and update all other benchmarks. If you successfully did that, post here and I will rename your old submission on Wilddash and you can upload a new submission. This is an exception and will not be repeated and does not mean other teams can request the same steps. Only submit to the benchmarks if you are satisfied/finished.

gepu0221 commented 2 years ago

Now I have updated all other benchmarks. Could you help me rename my old submission on Wilddash? Thanks!

Ok, please finalize your solution and update all other benchmarks. If you successfully did that, post here and I will rename your old submission on Wilddash and you can upload a new submission. This is an exception and will not be repeated and does not mean other teams can request the same steps. Only submit to the benchmarks if you are satisfied/finished.

ozendelait commented 2 years ago

I have renamed your old submission. Please upload your final version but add the word "rename" to the submission. I will then rename it to your RVC submission name. This is necessary as I'm doing a manual renaming and internal ids would otherwise collide if you submit under the same name right away.

gepu0221 commented 2 years ago

I have renamed your old submission. Please upload your final version but add the word "rename" to the submission. I will then rename it to your RVC submission name. This is necessary as I'm doing a manual renaming and internal ids would otherwise collide if you submit under the same name right away.

Thanks for your help! I have uploaded my final submission on the Wilddash adding the word "rename". Please help us update the submission to the leaderboard. Thanks!

ozendelait commented 2 years ago

Done. You only need to submit to Scannet to complete the challenge