Compatibility with MixMatch repo

[calmarazAM]

Hi!

Congrats on the paper, I'm excited to try FixMatch on my own dataset. I actually have previously used your code for MixMatch and I was wondering if this new repo is fully compatible with it, i.e., have there been any changes in the implementation of the common files between the two repos?

Thanks! :)

[david-berthelot]

A lot of code is shared but there are some differences that could be significant:

1. the datasets: the unlabeled dataset now also contains the labeled dataset with its labels removed (it was a space saving decision since we can use a single unlabeled set for all folds but it could also play in role in fixmatch performance).
2. the augmentation: we rewrote the pipeline since MixMatch to handle more complex augmentations such as RandAugment and CTAugment.

Note that MixMatch is also present in the FixMatch code base, so maybe working off the FixMatch code base would be a better alternative.

[moskomule]

Hi, I also noticed that you changed the way of weight decay from post_ops.extend([tf.assign(v, v * (1 - wd)) for v in utils.model_vars('classify') if 'kernel' in v.name]) (in MixMatch) to loss_wd = sum(tf.nn.l2_loss(v) for v in utils.model_vars('classify') if 'kernel' in v.name) (in FixMatch). Do you have any reasons for this change? Thanks!

[david-berthelot]

Because we changed the optimizer from Adam to momentum. L2 penalty doesn't play well with Adam in particular from what I understand and that's why we used weight decay back in MixMatch.

[moskomule]

Thanks. I didn't read that part in the paper. It's very helpful information.