search

paul007pl / MVP_Benchmark

MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration
https://mvp-dataset.github.io/
Apache License 2.0
120 stars 10 forks source link

Some problems about private test #15

Closed Dizzy-cell closed 2 years ago

Dizzy-cell commented 3 years ago
  1. Is the dataset for the private test the same as the dataset for the public test?
  2. How do we get feedback on the private test?
  3. What is the experimental environment of the private test (both software and hardware environment), do we need to configure it?
paul007pl commented 3 years ago

Hi. Thanks for asking.

  1. the private test dataset is not necessary to be the same with the public test, but of course we will evaluate your pretrained models on the public test dataset.
  2. we rank each method mainly based on the performance, and you will receive the final ranking list. Although it is not allowed to use multiple accounts, a few teams do not obey the rule. It is also not allowed if share the trained model or use very similar ideas with the other teams (plagiarism or duplicate submissions). We also need to check if a few teams cheat, and prevent using illegal or unfair operations (unfair or use extra data).
  3. Each participant can clearly state their environment (especially if they need extra 3-rd party libs), and make sure that a. it should not be very difficult to set the environment (hopefully). b. their methods can work well under the certain environement.
    We will use V-100 GPU to test. Hope everyone enjoy this competion, and our achievements can benefit CV community.
wangql16 commented 3 years ago

Hi, @paul007pl, When and how to upload the file related to private submission? I haven't found any notice about it.

paul007pl commented 3 years ago

:astonished: Really? I actually sent you emails two days ago via Codalab system... OK, I will use Gmail to send you again, please reply here once you receive. Thanks.

wangql16 commented 3 years ago

Oh, sorry, my bad. I forgot to check my email spam. Thanks for your reply.

paul007pl commented 3 years ago

No problem. I sent another one just now. Please have a check. :smiley:

wangql16 commented 3 years ago

Thanks for your help! I have received the email related to the private test. 😄

Dizzy-cell commented 3 years ago

“We rank each method mainly based on the performance, and you will receive the final ranking list. ” @paul007pl, When would we receive the final ranking list?