search

JuliaMath / MeasureTheory.jl

"Distributions" that might not add to one.
MIT License
386 stars 32 forks source link

RNGPartition from BAT #172

Open cscherrer opened 2 years ago

cscherrer commented 2 years ago

Today @oschulz showed me this code: https://github.com/bat/BAT.jl/blob/master/src/rngs/rng_init.jl

He suggested this could be split into a new package. If this happens, we should consider switching to it for reproducible RNGs for rand(::Chain). For now, this issue is just a placeholder so we don't forget.

oschulz commented 2 years ago

The way BAT uses it for reproducable MCMC is this: The MCMC chain initialier partitions it's RNG based on the maximum number of MCMC chains it will create:

rngpart = RNGPartition(rng, Base.OneTo(max_ncandidates)):

(https://github.com/bat/BAT.jl/blob/3fb68982df15c91a45c1d5b998bb0b39e57b5663/src/samplers/mcmc/chain_pool_init.jl#L63)

During construction, each MCMC chain partitions the RNG it's given (an element of the partition of the original RNG) again:

rngpart_cycle = RNGPartition(rng, 0:(typemax(Int16) - 2))

(https://github.com/bat/BAT.jl/blob/3fb68982df15c91a45c1d5b998bb0b39e57b5663/src/samplers/mcmc/mh/mh_sampler.jl#L104)

and each step in the MCMC chain runs

set_rng!(chain.rng, chain.rngpart_cycle, chain.info.cycle):

(https://github.com/bat/BAT.jl/blob/3fb68982df15c91a45c1d5b998bb0b39e57b5663/src/samplers/mcmc/mh/mh_sampler.jl#L161, https://github.com/bat/BAT.jl/blob/3fb68982df15c91a45c1d5b998bb0b39e57b5663/src/samplers/mcmc/mh/mh_sampler.jl#L237)

This way, each cycle of each MCMC chain get's it own independent RNG, all based on a single common seed. This makes the whole multi-chain MCMC result reproducible independent of exectution strategy/location (thread/process ID, etc.)