yaoyao-liu / mini-imagenet-tools

Tools for generating mini-ImageNet dataset and processing batches
https://mtl.yyliu.net/datasets/
MIT License
420 stars 57 forks source link

Are images sorted in the classes? #4

Open dsmic opened 5 years ago

dsmic commented 5 years ago

I use the files from the "Download Processed Images" (the tar file).

To me it seems that the images become sorted using MiniImageNetDataLoader, and the first images for each label are simpler to classify than later ones?

If I use shuffle = True at the end of the middle line (there are three occurrences of such lines)

  random.shuffle(sampled_character_folders)
  labels_and_images = self.get_images(sampled_character_folders, range(self.way_num), nb_samples=self.num_samples_per_class, shuffle=False)
  labels = [li[0] for li in labels_and_images]

it is much more difficult to train the data.

I got suspicious because the default in

def get_images(self, paths, labels, nb_samples=None, shuffle=True):

which is overwritten in the call.

yaoyao-liu commented 5 years ago

Thanks for your interest in this project.

If you use the default libraries of TensorFlow or PyTorch to build the deep neural networks, the order of samples in a single batch will be ignored. So the default setting for shuffle is False. You could set it True. It depends on your code.

If you hope to see an example of how this repo work in a real case, you may see: Meta-Transfer Learning.

dsmic commented 5 years ago

Thanks a lot for the very fast reply. I had an ugly error in my code which lead this assumption...

yaoyao-liu commented 5 years ago

Welcome. Feel free to create another issue if you have other questions.