how to recover performance when doing evaluation

[dlnewbie-h]

Dear authors

You mentioned that the model performance is evaluated using EMA of model parameters during training. So if we save a checkpoint during training and after training is done, we load that checkpoint and evaluate the model performance, wouldn't that be the performance of just one single checkpoint instead of the EMA of model parameters?

How can we deploy the model in real world setting say we want to saved the model checkpoint and then load the saved checkpoint to do inference on unseen data?

Thank you!

[carlini]

When we save a checkpoint, we save the EMA model weights. Then when you restore you get the restored EMA weights and can use those directly.

[dlnewbie-h]

Thanks for your response. Just to confirm what you mean, so you are saying the EMA model weights are already saved in the code, and when we use saver.restore(sess, save_path) we already get the EMA weights? Or are you saying people who uses the code should do it themselves?

[carlini]

Both are saved if I remember correctly. If you run

https://github.com/google-research/mixmatch/blob/1011a1d51eaa9ca6f5dba02096a848d1fe3fc38e/libml/train.py#L188-L198

then you should be able to see both the raw (no EMA) and EMA model predictions.

[dlnewbie-h]

In the code snippet what is the self.tune(16384) doing? why 16384?

Thanks again, I'm really interested in your work!