Implementation of WILDS-FMoW dataset in DomainBed

[saraalemadi]

Hi @lopezpaz,

I was wondering what do Region 0...5 proposed in the definition of WILDS-FMoW stands for? Are they mapped to the regions proposed in WILDS such as Asia, Europe, Americas, Africa, Oceania and Others? If so, in which order are they mapped?

Thanks.

Regards, Sara A. Al-Emadi

[piotr-teterwak]

Hi, I looked into the WILDS code (https://github.com/p-lambda/wilds/blob/472677590de351857197a9bf24958838c39c272b/wilds/datasets/fmow_dataset.py#L90).

it looks like it is

{0:'Asia', 1:'Europe', 2:'Africa', 3:'Americas', 4:'Oceania', 5:'Other'}

[saraalemadi]

Hi @piotr-teterwak,

Thanks for your reply. As far as I understand, FMoW-WILDS code defines domains in terms of years to address the DG problem (as shown in WILDS code https://github.com/p-lambda/wilds/blob/472677590de351857197a9bf24958838c39c272b/wilds/datasets/fmow_dataset.py#L77C13-L77C45). Whereas, regions are used to display the subpopulation shift (without explicitly splitting the data with respect to regions as described here: https://github.com/p-lambda/wilds/issues/138).

Therefore, could you kindly confirm if your implementation is aligned with FMoW-WILDS's definition for DG (i.e. domains are defined in terms of years)?

[piotr-teterwak]

Hi @saraalemadi , the DomainBed implementation does not use the years as domains, and instead uses regions. However, if you'd like to add another FMoW-WILDS-years dataset to the codebase and issue a pull request, it would be a great contribution!