BinaryRBM gets very noisy fantasy particles

[bhomass]

I ran through the MNIST BinaryRBM example with no modifications.

The output fantasy particles are noisy and lack variability.

Then I checked the output in the MNIST example and see that the same effects are there too. Noisy and no variability.

Next, I tried the fast_pcd, which is supposedly an improved sampling method. But it's even worse

Is there something that can be done to achieve better results? This is really not very useful as is.

[cossio]

The MNIST example (https://cossio.github.io/RestrictedBoltzmannMachines.jl/stable/literate/MNIST/) is running on Github CI so the training is limited. You can try increasing the number of Markov steps for each update to something like steps=50, which will make the training slower but is usually more robust.

Centering might also be helpful. Please see this example: https://github.com/cossio/StandardizedRestrictedBoltzmannMachines.jl/blob/master/example/mnist_notebook/stdrbm.ipynb.

Hope that helps!