Code reproducing figures from README

[kudkudak]

Thanks a lot for preparing this package!

I got recently interested in KFAC and have been trying to reproduce the deep autoencoder experiment. Would it be possible to share code that reproduces the figures in the README?

[duckworthd]

Hi @kudkudak,

I believe those experiments are here in the kfac/examples. If memory serves, I processed the log lines in Python to make the plots.

Let me know if the examples fail to generate similar-looking plots?

[duckworthd]

I'm sorry, the contents of the examples folder are definitely not the same as the ones in the plots! Code for those plots is still forthcoming.

If you're eager to reproduce sooner rather than later, the architecture is identical to the one published in Optimizing Neural Networks with Kronecker-factored Approximate Curvature

[vikramtankasali]

Hi @kudkudak , The code to reproduce the plots is here. There are couple of changes required to exactly reproduce the results. 1) Change momentum=0.0, momentum_type='qmodel', learning_rate=1. when creating the PeriodicInvCovUpdateKfacOpt. 2) Batch size schedule: Change batch size in load_mnist to maximum batch size value in the schedule. and feed into the batch_size placeholder appropriate value from the schedule in every iteration.

[kudkudak]

Thanks a lot! :)