search

tristandeleu / pytorch-meta

A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch
https://tristandeleu.github.io/pytorch-meta/
MIT License
1.97k stars 256 forks source link

Limiting the amount of generated tasks #132

Closed ojss closed 2 years ago

ojss commented 3 years ago

Hi @tristandeleu and team!

I wanted to know if there is a way I can limit the number of tasks being generated. I want to add a validation step with say about 500 tasks, I am not able to find a way to limit the generated tasks. Please let me know if it is possible.

Thank you for the great work!

tristandeleu commented 3 years ago

I'm sorry for the late reply, and thank you for the kind words! All the datasets are valid PyTorch's Dataset, so you can use all the features from PyTorch here as well, including Subset. This is something I'm using a lot myself, especially to have a fixed meta-validation set. The only difference is that now the indices must be tuples of classes (in order to be a valid task). For example

meta_val_dataset = miniimagenet(folder, shots=5, ways=5, shuffle=True, test_shots=15, meta_val=True, download=True)
meta_val_indices = [(8, 0, 10, 1, 12), (9, 2, 6, 3, 15), (13, 6, 12, 11, 3), (1, 6, 4, 14, 13), ...]
meta_val_dataset = Subset(meta_val_dataset, meta_val_indices)

Hope this helps!

ojss commented 3 years ago

Thanks a lot this works, it is really helpful!