What percentage of domainbed are you using?

[miladsoltany]

Dear Authors,

I appreaciate your exciting work. I was going through your code and realized that for domainbed datasets including pacs, you do use 15% of the target domain. However, it was my understanding that you train the target client (server) using that 15% and evaluate on the remaining 85% of that specific domain. I noticed that this is not the case and accoridng to line 409 in file /main_domain_bed.py, you test on server_dls['test'][0], which only contains 360 images for the art_painting class of pacs which includes 2048 images.

I would appreaciate it if you could kindly shed some light on this matter.

Best, Milad

[enyijiang]

Hi,

Thanks for your question. We train the target client using 15% and report the performance on the test dataset, which is split from the whole dataset beforehand (line 275) with a holdout fraction of 20%. Also, testing on the rest of 85% is possible, which may not affect the results too much.

Thanks, Authors