Closed romain-xu-darme closed 1 year ago
The validation set for CIFAR-10 as ID have 2 parts: CIFAR-10's val as ID and CIFAR-100's val as OOD. The goal is to find the optimal hyperparam for OOD detection, so we have validation data from both ID and OOD set.
Thank you. Now I understand ... but then why is CIFAR100 also using val_cifar100 as OoD validation set in cifar100_ood.yml ?
You are correct. cifar100_ood should use val_cifar10.txt I will double-check with my teammates on this issue. Thank you for pointing it out. Nevertheless, the problem shall not affect too much on our main results and conclusion, since if different hyperparam makes no difference on val, we will take default hyperparam for the result. Anyhow, we will look into this issue. Thanks!
You're welcome :)
Hello, thanks a lot for developing this library! It seems that there is an issue in the OoD configuration for CIFAR10: in configs/datasets/cifar10/cifar10_ood.yml, shouldn't the validation set point to cifar10 rather than cifar100? Thanks