Try model averaging. This is reported to significantly increase the test performance of generative models:
Ployak averaging, applied only at test time. Train as per usual.
Stochastic weight averaging. See here for pytorch implementation.
Hypothesis
The performance of the model at test time will be much better with an averaged model. People who KNOW say as much.
Results
Write up the results of your experiment once it has completed and has been analysed. Include links to the treatment run, and also to the baseline if appropriate.
Acceptance Criteria
[ ] Implement Polyak averaging and apply only at test time.
What
Try model averaging. This is reported to significantly increase the test performance of generative models:
Hypothesis
The performance of the model at test time will be much better with an averaged model. People who KNOW say as much.
Results
Write up the results of your experiment once it has completed and has been analysed. Include links to the treatment run, and also to the baseline if appropriate.
Acceptance Criteria