search

mariodoebler / test-time-adaptation

A repository and benchmark for online test-time adaptation.
MIT License
188 stars 16 forks source link

mixed domains TTA setting #8

Closed Fenghao1995 closed 1 year ago

Fenghao1995 commented 1 year ago

Nice work for universal TTA. But I have a little problem. For ImageNet-C, how to keep per corruption with 50000 samples in the mixed domains TTA setting. By setting "_C.CORRUPTION.NUM_EX = -1", we can only use 75,000 (5000 per corruption *15 )samples.

RobMarsden commented 1 year ago

Hi @Fenghao1995,

thank you for your interest in our work. If you want to work with IN-C in the "mixed_domains" setting with the full 50000 samples per domain, just set CORRUPTION.NUM_EX = 750000. (Short explanation: If you set CORRUPTION.NUM_EX != -1 for IN-C, the 50k sequence will be loaded instead of the default 5k sequence from robustbench. Since the "mixed_domains" setting is taken into account, all 15 * 50k = 750k samples are loaded first before the test sequence is shortened to the desired length).

All the best, Rob

Fenghao1995 commented 1 year ago

Thanks for your quick reply!