search

ynysjtu / ood_bench

47 stars 8 forks source link

Naming inconsistency of the CelebA experiment: "CelebA_Blond" or "CelebA_Blonde" ? #2

Closed alexrame closed 2 years ago

alexrame commented 2 years ago

Thank you for the great work and for open-sourcing your benchmark. I believe there is a naming inconsistency for your CelebA experiment.

The dataset is named "CelebA_Blond" here https://github.com/ynysjtu/ood_bench/blob/cac82a5ea97a9d4772066fe40a17d7db185e9615/DomainBed/domainbed/datasets.py#L53 .

In contrast, you define hyperparameter distributions for the "CelebA_Blonde" dataset (with an "e" at the end) here https://github.com/ynysjtu/ood_bench/blob/cac82a5ea97a9d4772066fe40a17d7db185e9615/DomainBed/domainbed/hparams_registry.py#L136

So overall, you may need to fix this. Moreover, could this affect the results reported in your paper? Thanks in advance

m-Just commented 2 years ago

Thank you for letting us know! We've fixed this in commit 0e5d6c. We also checked the history and the conclusion is that it does not affect the results reported in our paper. "CelebA_Blonde" was the original one and later we changed it to "CelebA_Blond" for simplicity during code cleanup after the experiments.