search

facebookresearch / DomainBed

DomainBed is a suite to test domain generalization algorithms
MIT License
1.41k stars 299 forks source link

Extension to Unsupervised Domain Adaptation #28

Closed daysm closed 3 years ago

daysm commented 3 years ago

Thank you for your research and work on DomainBed.

In Appendix E.1 you mention the possibility of extending the method .update(minibatches, unlabeled) to accept a minibatch of unlabeled examples.

Could you explain how you think this extension would be implemented best? Would unlabeled specify whether minibatches is unlabeled? I'm working on extending the .update method of AbstractDANN for UDA.

lopezpaz commented 3 years ago

What I had in mind is redefining all update methods to .update(minibatches, unlabeled=None). When unlabeled=None, DomainBed operates as usual (domain generalization). Otherwise, unlabeled is expected to contain a minibatch of unlabeled examples from the test domain, and it is up to each of the algorithms to leverage it to their advantage (domain adaptation).

The second change would be to add a flag --domain_adaptation to train.py, which would split the test loader into the unlabeled examples used for training and the actual test set used for evaluation (and oracle model selection).

lopezpaz commented 3 years ago

Done in b953488d4dcfcc76427f07958b133b87d24a48e5.