tristandeleu / pytorch-meta

A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch
https://tristandeleu.github.io/pytorch-meta/
MIT License
1.98k stars 256 forks source link

Purpose of Classsplitter is unclear #127

Closed jaeho3690 closed 3 years ago

jaeho3690 commented 3 years ago

Hi, I am quite confused about the purpose of the method Classsplitter. Below is an example code. The thing I don't get is that the dataset instance is downloaded with the argument meta_train turned true. Why is the dataset that is downloaded for the purpose of training is split once again? Is the ClassSplitter used to split the training set into train-test set once again?

dataset = torchmeta.datasets.MiniImagenet("data", num_classes_per_task=5,
meta_train=True, download=True)
dataset = torchmeta.transforms.ClassSplitter(dataset, num_train_per_class=1,
num_test_per_class=15, shuffle=True)

Is there anything wrong with writing the code like below?

    if config.dataset =='mini':
        normalize = Normalize(mean=[0.485, 0.456, 0.406],std=[0.229, 0.224, 0.225])
        train_dataset = miniimagenet(config.data_root,shots=config.n_shot,ways=config.n_way,shuffle=True,test_shots=1,
                                    meta_split='train',download=True,transform=Compose([ToTensor(),normalize]))
        test_dataset = miniimagenet(config.data_root,shots=config.n_shot,ways=config.n_way,shuffle=True,test_shots=1,
                                    meta_split='test',download=True,transform=Compose([ToTensor(),normalize]))
    train_loader = BatchMetaDataLoader(train_dataset,batch_size=config.meta_batch_size,shuffle=True,num_workers=4)
    test_loader = BatchMetaDataLoader(test_dataset,batch_size=config.meta_batch_size,shuffle=True,num_workers=4)
jaeho3690 commented 3 years ago

After going through https://github.com/tristandeleu/pytorch-meta/blob/c84c8e775f659741f7ad2ab9fbcfc1a78a4e76c9/torchmeta/datasets/helpers.py#L19 I found out that the helper function loads the same dataset but with the class splitter and conversion of label. I am really thankful for your work and I think the library can be used by more people if these small details can be included in the Readme file.