how to create a custom dataset

[raymond00000]

I could not figure out how to create a custom dataset.

I have a dataset that return (x, target), let it called "trainset" I have a collate function, let it called "collate_fn"

so, I tried:

metaDataLoader = MetaDataLoader(trainset, collate_fn=collate_fn) bmetaDataLoader = BatchMetaDataLoader(metaDataLoader) for batch in bmetaDataLoader: print(batch)

but it returned

data = [self.dataset[idx] for idx in possibly_batched_index]

TypeError: 'MetaDataLoader' object does not support indexing

[tristandeleu]

The datasets used in Torchmeta, and inside a MetaDataLoader are inherited from the MetaDataset class. There is no way at the moment to automatically transform a standard PyTorch dataset into a MetaDataset, because it really depends on the way you want to formulate your problem (is it a few-shot regression problem?, is it a few-shot classification problem?).

In order to create your own dataset, I suggest you to look at some of the existing datasets for inspiration:

• SinusoidTask if you have a regression problem. This requires you to create a MetaDataset class and a Task class.
• Omniglot if you have a few-shot classification problem. This requires you to create a ClassDataset class (to generate the dataset of all examples for a single label), and a Task. The CombinationMetaDataset automatically combines multiple labels together to create a classification task.

Look also at https://github.com/tristandeleu/pytorch-meta/issues/102#issuecomment-726741999 for a more detailed description of the inner-workings.

I hope this helps!