Reproducible Emulator training - Githubissues

CliMA / CalibrateEmulateSample.jl

Stochastic Optimization, Learning, Uncertainty and Sampling

https://clima.github.io/CalibrateEmulateSample.jl/dev

Apache License 2.0

81 stars 14 forks source link

Reproducible Emulator training #266

Closed odunbar closed 7 months ago

odunbar commented 7 months ago

Currently the full CES pipeline "Should" be deterministic, given the following rngs are passed:

(Data noise seed when conducting a perfect experiment)
C: calling construct_initial_ensemble(rng, ...) or building EnsembleKalmanProcess(...,rng = rng)
E: building ScalarRandomFeatureInterface(...,rng=rng) or VectorRandomFeatureInterface(...,rng=rng)
S: calling optimize_stepsize(rng, mcmc,...) or sample(rng, mcmc,...)

It appears that some part of the Emulator training framework is still subject to randomness (e.g. also for GP) making it 'slightly' non-reproducible.

odunbar commented 7 months ago

Seems like this was resolved, it was a script causing the irreproducibility