search

eccv22-ood-workshop / ROBIN-dataset

ECCV 2022 Workshop: A Benchmark for Robustness to Individual Nuisances in Real-World Out-of-Distribution Shifts
http://www.ood-cv.org/
46 stars 6 forks source link

IID and OOD ? #17

Open wujiekd opened 2 years ago

wujiekd commented 2 years ago

If I understand it correctly, the better the OOD and the worse the IID, the better the game

DTennant commented 2 years ago

Hi,

Thanks for your interest in our challenge. We only penalize submissions that have an IID performance higher than a threshold, we do not use the absolute value of IID performance for ranking, the ranking will only be based on the average OOD performance on the six nuisances.

wujiekd commented 2 years ago

Thak you for your reply~

wujiekd commented 2 years ago

Hi,

Thanks for your interest in our challenge. We only penalize submissions that have an IID performance higher than a threshold, we do not use the absolute value of IID performance for ranking, the ranking will only be based on the average OOD performance on the six nuisances.

By the way, where is the baseline, thanks ~