search

tristandeleu / pytorch-meta

A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch
https://tristandeleu.github.io/pytorch-meta/
MIT License
1.97k stars 256 forks source link

Reproducibility is broken with class augmentations #56

Closed marcociccone closed 4 years ago

marcociccone commented 4 years ago

Hi! Even if you fix the random seed everywhere, when you use class augmentations such as Rotation([90, 180, 270]), results are not always the same. It took me a while, but I found that it is because class augmentations become unique using a set(), here and this is not stable. The order of the augmentations might not be the same at different runs!

tristandeleu commented 4 years ago

Hi! That's a very good point, thank you. It should probably be just a matter of switching set() for OrderedSet().

marcociccone commented 4 years ago

Yes, you're right, but I only found an external pip package OrderedSet. Or if you prefer I can find an alternative. I can test it and submit a PR if that is ok for you.

tristandeleu commented 4 years ago

I think this additional dependency doesn't hurt too much, and would probably be easiest solution. That would be great if you could open a PR with it, thank you!