search

xyutao / fscil

Official repository for Few-Shot Class-Incremental Learning (FSCIL)
226 stars 37 forks source link

Test set #12

Closed akashgokul closed 4 years ago

akashgokul commented 4 years ago

For datasets like CUB, do you have a test.txt? If not, how did you gather your test dataset (is it all the test samples for each class in each session)?

akashgokul commented 4 years ago

Could you push a test_session_i.txt for each session?

xyutao commented 4 years ago

Hi @akashgokul , The test splits for CUB200 have been uploaded to the folder data/cub200. The list of the test images for each session's classes is stored in the file "test_i.txt". You may notice that when evaluating the model at session t, please accumulate all the test images of the encountered sessions, from "test_1.txt" to "test_t.txt".

The original train-test split of CUB200 has been given by the official dataset files. You may download CUB200, unzip it, put it under the folder "data", and then run the split script "data/cub200_split.py" to reproduce the above test splits.

akashgokul commented 4 years ago

Okay thank you so much!

I'm sorry to keep bothering but could you push the test_i.txt for the cifar100 and miniImageNet datasets? Sorry to keep asking, but this makes reproducing/comparision using other libraries/methods much easier.

xyutao commented 4 years ago

Hi, the test_i.txt of cifar100 and miniImageNet have been generated and pushed. Please check out the updates.

akashgokul commented 4 years ago

Is there a reason that for CUB200, test_1.txt does not include the 100th class but session_1.txt does? Is this supposed to be range(start_class, end_class + 1)?

xyutao commented 4 years ago

Sorry, it's a bug. Now it's fixed. Thanks for pointing it out.

akashgokul commented 4 years ago

Thank you for the help!

Did you have this error when reporting your accuracies in the paper and README?

xyutao commented 4 years ago

No. The results in the paper and README are evaluated using the dataloader script, which is correct. The provided text files are generated alone from a new script cub200_split.py which was written several days ago for ease of use.