zhiCHEN96 / ConceptWhitening

MIT License
118 stars 27 forks source link

Exact version of Places365 #8

Open expectopatronum opened 3 years ago

expectopatronum commented 3 years ago

Hi! I am trying to follow the instructions in the README and unfortunately I am stuck already when downloading the Places365 dataset.

There are several different versions available and I don't know which one to take. From the root directory name data_256 I assume you are using the small images (256 256) and not the high resolution version. Since you are using the same/similar file structure as in the .tar file below "Small images (256 256) with easy directory structure" I downloaded this, but here the test data is missing. Is this the correct train/validation data? What is the correct test data?

Thanks and best regards Verena

zhiCHEN96 commented 3 years ago

Hi Verena,

The test dataset is here (just one line above, under the Small images (256 * 256) section). However, the label of test data is not provided, because the Places365 dataset was once a competition, and the organizers do not want people to overfit the test data. However, they do have an evaluation server to evaluate the test accuracy.

For research purpose, what you can do is to use the validation set for testing, and randomly move 10%-20% training data to another folder as validation set. Note that, you should only tune your parameters based on the validation accuracy instead of test accuracy.

Best, Zhi

expectopatronum commented 3 years ago

Hi, thanks for the clarification, it was not really clear from the README. I would like to reproduce the results from your paper, so I would like the split to be as close as possible to the one you used. Do you still have the exact split you used? Or at least the exact percentage of examples from the training data you used for validation?

Best regards Verena

zhiCHEN96 commented 3 years ago

Hi, I used 20% of training data for validation.