search

eccv22-ood-workshop / ROBIN-dataset

ECCV 2022 Workshop: A Benchmark for Robustness to Individual Nuisances in Real-World Out-of-Distribution Shifts
http://www.ood-cv.org/
46 stars 6 forks source link

When will the dataset and baseline for 3D pose estimation be released #7

Open Breeze-zyuhan opened 2 years ago

Breeze-zyuhan commented 2 years ago

Hi, When will the dataset and baseline for 3D pose estimation be released? Thx.

DTennant commented 2 years ago

Hi, Thanks again for your interest in our workshop, and we apologise for the late update. I have update the README in this repo and update a link to the full test set and labels in the README, the codalab submission server will also be online within this week.

Here are some changes to the competition:

  1. The Phase-1 of the competition will not be a code submission challenge, we have released all the test data and labels in this repo. And Phase-1 will last longer than original planed, we will ask each team to provide a description of their developing environment at the end of Phase-1, Phase-2 will still be code submission challenge.
  2. We will be using Top-1 accuracy for image-classification, mAP@50 for object detection, and Acc@pi/6 for pose estimation as the metric, the IID test performance will also be considered as per request of the sponsor, we will penalize submissions that are significantly different in IID performance with our baseline.
  3. The only limitation now is that the model should only be trained on the given training set and/or the ImageNet-1k dataset, no additional dataset is allowed. You can use any ensemble, data augmentation, or test-time training techniques.

Thanks again for your patience.

Best regards

Breeze-zyuhan commented 2 years ago

Thanks for your response! BTW, when will the specific schedule be announced?

DTennant commented 2 years ago

Hi, We have just released the codalab server for evaluation, and confirmed the schedule with our sponsor, you can check it out here http://www.ood-cv.org/challenge.html