search

MelJan / PyDeep

PyDeep is a machine learning / deep learning library with focus on unsupervised learning. The library has a modular design, is well documented and purely written in Python/Numpy. This allows you to understand, use, modify, and debug the code easily. Furthermore, its extensive use of unittests assures a high level of reliability and correctness.
http://pydeep.readthedocs.io/en/latest/index.html
52 stars 14 forks source link

centering fails miserably on mnist data #11

Open bhomass opened 1 year ago

bhomass commented 1 year ago

Is centering supposed to be a delicate operation? I tested out the mnist example with update_offsets = 0.01 the vis prob immediately diverges

Epoch Recon. Error Log likelihood Expected End-Time vis = (-0.9625000342860309, 1.077954957023838) vis = (-1069.8595390064602, 1327.4006511194739) vis = (-53567281.491539374, 66476566.5927559) vis = (-4938184234886.023, 6128247022497.816) vis = (-4.5004117525694554e+17, 5.584974883656288e+17) vis = (-6.1943772698263585e+22, 7.687172501964909e+22) vis = (-1.128616196508766e+28, 1.4006036463642153e+28) vis = (-2.0463202112046235e+33, 2.5394669669882995e+33) vis = (-4.604357086551687e+38, 5.713970209302338e+38) vis = (-1.0258660997795336e+44, 1.2730916005612099e+44) vis = (-2.7208673874176135e+49, 3.3765746016041543e+49) vis = (-7.146736158979544e+54, 8.869042243796985e+54) vis = (-2.1715736576845397e+60, 2.6949054893151322e+60) vis = (-7.46573948398941e+65, 9.264922811161907e+65)

soon the values are all nan.

without update_offset, the model runs just fine with really good sampling results. But, there is a whole paper on the merit of centering RBM, so why doesn't it work with MNIST data?

MelJan commented 1 year ago

Please provide the code, otherwise it is pretty hard to guess where the issue lies.