search

p-lambda / wilds

A machine learning benchmark of in-the-wild distribution shifts, with data loaders, evaluators, and default models.
https://wilds.stanford.edu
MIT License
551 stars 128 forks source link

Question about the creation of WILDS-FMoW subset #147

Closed saraalemadi closed 1 year ago

saraalemadi commented 1 year ago

Hi,

In your paper, you have mentioned that you have used a subset of FMoW. However, in the rgb_metadata.csv file provided, you analyse the entire fmow dataset and I couldn't find where in the code you are creating the subset (sampling from the rgb_metadata.csv file). I have also looked at the parameter frac which was equal to 1.0 in the config file as well as the worksheet (https://worksheets.codalab.org/rest/bundles/0x20182ee424504e4a916fe88c91afd5a2/contents/blob/log.txt). Therefore, I would greatly appreciate it if you could kindly let me know how you created the subset.

Thank you.

Sara A. Al-Emadi

kohpangwei commented 1 year ago

Hi Sara,

I'm sorry for the late reply. If you see our description in Appendix E.7 (https://arxiv.org/pdf/2012.07421.pdf), we filter for images in the appropriate time range, so we don't use all of the images in the original FMoW dataset. Hope that helps.