david-berthelot
commented
5 years ago This implementation does not use pre-trained networks. There's no code mechanisms specifically designed for transfer learning in this code base.
That being said, you could do pre-init without any coding use the current code.
- You could first train a neural network using fully supervised on some dataset A,
- Then copy the checkpoint in another folder when using MixMatch for training on dataset B.
Basically the checkpoints do no contain the data, they contain the networks weights and other variables (steps, running averages, etc...). So you can do some clever file renaming / copying to simulate pretraining.
varunnair18
Does this implementation use pre-trained weights from any dataset to achieve the results shown in the paper?
I know that in Oliver et. al., 2018 ("Realistic Evaluation of Deep Semi-Supervised Learning Algorithms"), which is cited by your paper, they claim that transfer learning or using pre-trained weights may actually lead to better results than using semi-supervised learning.
I'm curious to know if this implementation does use transfer learning, or has a mechanism for doing so. If so, then it would allow us to see if combining MixMatch with transfer learning has even more performance improvements and how well using transfer learning alone performs as well.