log_pseudolikelihood for GaussianRBM

[bhomass]

has the GaussianRBM been tested with MNIST data the same way as the MNIST example for BinaryRBM?

When I created a notebook to run a test in the exact same fashion as the example code, it blows up on # println("log(PL) = ", mean(@time log_pseudolikelihood(rbm, train_x)))

and indeed, in the pseudolikelihood.jl file, there is no signature for substitution_matrix_sites() to accept rbm::RBM{<:Gaussian}

[cossio]

pseudolikelihood is not implemented for Gaussian visible layer. However for GaussianRBM, the likelihood can be computed exactly (no need for the pseudo likelihood approximation). This is implemented, see log_likelihood.

[bhomass]

thanks this works.

I loaded the real value MNIST data into GaussianRBM train_x = Array{Float}(train_x[:, :, train_y .== 0]) rbm = GaussianRBM((28,28), tuple(400)) and ends up with this error AssertionError: all((>)(0), λ) rescale_activations!(layer::RestrictedBoltzmannMachines.Gaussian{1, Matrix{Float64}}, λ::Vector{Float64}) @ RestrictedBoltzmannMachines ~/RestrictedBoltzmannMachines.jl/src/gauge/rescale_hidden.jl:46

have you seen this before?

[cossio]

Does this error happen after training? Can you post a full example? Thanks

[bhomass]

during training on pcd! call.

I followed the same process as in the mnist example, just change to real numbers.

Float = Float32
train_x = MLDatasets.MNIST(split=:train)[:].features
train_y = MLDatasets.MNIST(split=:train)[:].targets
train_x = Array{Float}(train_x[:, :, train_y .== 0])

rbm = GaussianRBM((28,28), tuple(400))
initialize!(rbm, train_x) # match single-site statistics

batchsize = 256
iters = 10000
history = MVHistory()
@time pcd!(
    rbm, train_x; iters, batchsize
)

AssertionError: all((>)(0), λ)

Stacktrace: [1] rescale_activations!(layer::RestrictedBoltzmannMachines.Gaussian{1, Matrix{Float64}}, λ::Vector{Float64}) @ RestrictedBoltzmannMachines ~/RestrictedBoltzmannMachines.jl/src/gauge/rescale_hidden.jl:46 [2] rescale_hidden!(rbm::RestrictedBoltzmannMachines.RBM{RestrictedBoltzmannMachines.Gaussian{2, Array{Float64, 3}}, RestrictedBoltzmannMachines.Gaussian{1, Matrix{Float64}}, Array{Float64, 3}}, λ::Vector{Float64}) @ RestrictedBoltzmannMachines ~/RestrictedBoltzmannMachines.jl/src/gauge/rescale_hidden.jl:10 [3] rescale_weights!(rbm::RestrictedBoltzmannMachines.RBM{RestrictedBoltzmannMachines.Gaussian{2, Array{Float64, 3}}, RestrictedBoltzmannMachines.Gaussian{1, Matrix{Float64}}, Array{Float64, 3}}) @ RestrictedBoltzmannMachines ~/RestrictedBoltzmannMachines.jl/src/gauge/rescale_hidden.jl:25 [4] pcd!(rbm::RestrictedBoltzmannMachines.RBM{RestrictedBoltzmannMachines.Gaussian{2, Array{Float64, 3}}, RestrictedBoltzmannMachines.Gaussian{1, Matrix{Float64}}, Array{Float64, 3}}, data::Array{Float32, 3}; batchsize::Int64, iters::Int64, wts::Nothing, steps::Int64, optim::Optimisers.Adam{Float32}, moments::Array{Float32, 3}, l2_fields::Int64, l1_weights::Int64, l2_weights::Int64, l2l1_weights::Int64, zerosum::Bool, rescale::Bool, callback::Returns{Nothing}, vm::Array{Float64, 3}, shuffle::Bool, ps::NamedTuple{(:visible, :hidden, :w), Tuple{Array{Float64, 3}, Matrix{Float64}, Array{Float64, 3}}}, state::NamedTuple{(:visible, :hidden, :w), Tuple{Optimisers.Leaf{Optimisers.Adam{Float32}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Tuple{Float32, Float32}}}, Optimisers.Leaf{Optimisers.Adam{Float32}, Tuple{Matrix{Float64}, Matrix{Float64}, Tuple{Float32, Float32}}}, Optimisers.Leaf{Optimisers.Adam{Float32}, Tuple{Array{Float64, 3}, Array{Float64, 3}, Tuple{Float32, Float32}}}}}) @ RestrictedBoltzmannMachines ~/RestrictedBoltzmannMachines.jl/src/train/pcd.jl:68 [5] top-level scope @ ./timing.jl:273 [inlined] [6] top-level scope @ ./In[19]:0