Stochastic reconfiguration for autoregressive models?

[NnktYoshioka]

Hi,

Thank you very much for making this great library public! I was walking through the example codes and noticed that the stochastic reconfiguration is available, e.g. for Rbm. However, when I tried to apply this to convnet autoregressive model, I encountered an error.

Concretely, I executed a code looking like the following, but the optimizer could not be defined. Is SR simply not available so far, or is there a workaround?

Thanks very much in advance!

Nobu

from tensorflow.keras.layers import Input
from tensorflow.keras.models import Model

from flowket.optimizers import ComplexValuesStochasticReconfiguration
from flowket.layers import LogSpaceComplexNumberHistograms
from flowket.machines import ConvNetAutoregressive2D
from flowket.operators import Ising
from flowket.optimization import VariationalMonteCarlo, loss_for_energy_minimization
from flowket.samplers import AutoregressiveSampler

Nx = 3
Ny = 3

depth = 2
nchannel = 8

# Define machines
hilbert_state_shape = [Nx, Ny]
inputs = Input(shape=hilbert_state_shape, dtype='int8')
convnet = ConvNetAutoregressive2D(inputs, depth=depth, num_of_channels=nchannel, weights_normalization=False)
predictions, conditional_log_probs = convnet.predictions, convnet.conditional_log_probs
predictions = LogSpaceComplexNumberHistograms(name='psi')(predictions)

model = Model(inputs=inputs, outputs=predictions)
conditional_log_probs_model = Model(inputs=inputs, outputs=conditional_log_probs)
optimizer = ComplexValuesStochasticReconfiguration(model, convnet.predictions_jacobian, lr=0.05, diag_shift=0.1,
                                                   iterative_solver=False, )

[noamwies]

Hi, Currently the SR implemented only for networks with complex value weights e.g. ComplexValuesSimpleConvNetAutoregressive1D,Rbm By using ComplexConv2D & lncosh It should be simple to implement complex weights variant of ConvNetAutoregressive2D

Noam

[noamwies]

Let me know if this solve the problem

[NnktYoshioka]

Thanks for introducing me to 1d autoregressive Convnet. As you suggested, I also got to define 2d model by combining ConvNetAutoregressive2D and ComplexValuesSimpleConvNetAutoregressive1D.