Open alex-service-ml opened 5 years ago
I see. You are correct. I will amend the dataset_tools.py file to ensure that new datasets are treated differently.
As for your second question, since I am using dictionaries to store the samples, in theory it should be able to handle classes of variable amounts of samples.
Just updated the dataset_tools.py file. Have another go at this and let me know if it works.
Wow, thank you. I'll be giving it another try today and I'll let you know!
I identified a couple more issues with using a custom dataset. I decided to use Fashion MNIST, which is grayscale (28, 28, 1), but load_image
in data.py
defaults to converting to 3-channel, which then causes shape issues during the forward pass. In a similar vein, get_transforms_for_dataset
doesn't return anything if using a custom dataset name. Fixing these two issues, it seems fashionmnist has kicked off training.
A couple more issues in generate_configs.py
: must create a relevant entry in hyper_config_dict
and add a relevant elif
entry to assign search_name
.
Hello, I know it's been a while but if you managed to get a custom dataset working, would you mind explaining the changes you had to make? I'm a bit confused on what exactly the entries in hyper_config_dict
refer to and how I should adjust them to my dataset. Also, how did you adjust load_image
to account for a single channel image?
maybe_unzip_dataset
inHowToTrainYourMAMLPytorch/utils/dataset_tools.py
doesn't appear to support datasets other than omniglot and mini-ImageNet. I have created my own dataset (following the two-layer directory structure) with 100 classes. Following the instructions in the README, I created a config for the custom dataset(+experiment) to run. I then generated the config and the script, then attempted to run the script. I'm met with the following error:I suspect this section to be the issue, as it seems my dataset gets counted, then deleted, then the function calls itself again and obviously no longer sees the dataset.
In addition to the issue above, I wanted to ask: do all tasks/classes have to have the same number of samples for training, e.g. 600? Or can one task have 600 samples and another have 257?