ReactiveBayes / RxInfer.jl

Julia package for automated Bayesian inference on a factor graph with reactive message passing
MIT License
270 stars 23 forks source link

LSTM example output is different between local version and documentation version #42

Closed bvdmitri closed 1 year ago

bvdmitri commented 1 year ago

The output of the prediction in the generated version in the documentation is different from the one, which I get locally: https://biaslab.github.io/RxInfer.jl/stable/examples/Kalman%20filter%20with%20LSTM%20network%20driven%20dynamic/

image

vs

image

@Chengfeng-Jia do you have an idea why it might happen?

Chengfeng-Jia commented 1 year ago

These two figures represent two different examples. In the "Generate data" section, there are some randomness(push!(data,nfd+randn(3)) is used to add observation noise, and shuffle!(new_order) is used to shuffle the data sets). Therefore, each time you run this example, the observations and ground truth will be different. The weights of neural network have not changed, but the inputs have changed. I still think these predictions makes sense, even there are new generated data. These data are generated by the same equations, i.e., Lorenz system. The neural network is expected to capture the shared dynamics of these different samples.

bvdmitri commented 1 year ago

In the beginning of the notebook I can see the Random.seed!() call, results should no be different. I also always get the same result if I rerun the notebook locally. If I compare the data it also looks identical to me (check visually). I this is the case, indeed, that results are changing from run to run that must be fixed. There is a library called StableRNGs that we use to generate identical data across all versions of Julia.

bvdmitri commented 1 year ago

Fixed in #43