search

ybendou / easy

This repository is the official implementation Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients.
MIT License
112 stars 18 forks source link

How to test features on tieredimagenet #6

Closed Nan-S closed 2 years ago

Nan-S commented 2 years ago

Hello,

May I know how to run test for tieredimagenet using ASY? I used the following command line, but it seems not working for multiple GPU. python main.py --dataset-path 'my data path' --dataset tieredimagenet --model resnet12 --test-features tieredfeaturesAS1.pt11 --preprocessing ME --n-shots 1 --device cuda:0123 If I just used --device cuda:0, I got out of memory error as the feature is too big for tieredimagenet. I am using 4 2080ti GPU with 12GB memory per device.

ybendou commented 2 years ago

Hello,

Thanks for pointing this out. The reason it doesn't work is because the features tensor with tieredimagenet is too big. You need to change the batchsize of the few shot runs. Normally it was fixed to 100. I just pushed a new version where you can change it in the command line by using the new argument --batch-fs. Can you first do a pull and then test the following command? git pull python main.py --dataset-path 'my data path' --dataset tieredimagenet --model resnet12 --test-features tieredfeaturesAS1.pt11 --preprocessing ME --n-shots 1 --device cuda:0123 --batch-fs 20 If 20 is still too big, try with a smaller value.

Let me know if it's still not working.

Best,

Nan-S commented 2 years ago

Hello,

Thanks for your prompt reply. The following command works for me when I only use 1 GPU. python main.py --dataset-path 'my data path' --dataset tieredimagenet --model resnet12 --test-features tieredfeaturesAS1.pt11 --preprocessing ME --n-shots 1 --device cuda:0 --batch-fs 10

I still got an error if I use --device cuda:0123, it seems the code not support muti-gpu testing now.

File "main.py", line 318, in test_features = torch.cat([torch.load(fn, map_location=torch.device(args.device)).to(args.dataset_device) for fn in filenames], dim = 2) RuntimeError: Invalid device string: 'cuda:0123'

Best,

ybendou commented 2 years ago

Hello,

Yes you're right, the code doesn't support multi-gpu when doing few-shot evaluation. It only supports it during training for now. We will include multi-gpu for testing in an upcoming version.

I will close this thread for now.

Best,