search

drckf / paysage

Unsupervised learning and generative models in python/pytorch.
Other
119 stars 25 forks source link

Heat capacity training metric #73

Closed d-rams closed 7 years ago

d-rams commented 7 years ago

The heat capacity is a good metric for determining if an RBM is in a {glassy, ferromagnetic, paramagnetic} phase, and it's change during training can indicate if the model is approaching a phase transition.

I'd like to implement the capacity via MCMC sampling as well as via the TAP3/4 formulas.

Once we can estimate the phase during training we can perhaps change or even switch training methods dynamically according to which hyperparameters work best in which phases.

d-rams commented 7 years ago

Heat capacity as the second cumulant of the joint energy is in a PR