Open guillochon opened 7 years ago
Hi,
I think a smarter acor is a good idea. I have more datasets and when I run MCMC for them in a loop, for some of them I can compute autocorrelation time with c = 10, but for some with c = 1, and all the values in between.
Hi all,
So I have a (set of) problems with long (but highly variable) burn-in times that are not known apriori. Minimization during the burn-in phase can get you close to a burned-in state, but whether one is burned in or not is not reliably known.
It appears that if you are not burned in and run
acor
over the full length of the chain, there is an extremely high probability of failure unlessc
is very small (I've found that often onlyc = 1
works reliably). A simple way to avoid this issue is to only calculateacor
beyond some "burned in" step, which I've done here in a modifiedget_autocorr_time
(via a new parametermin_step
):But what value should one choose for
min_step
? (If we knew, we'd know when we were burned in!) It seems to solve this problem, one would like to be able to runacor
withc
set as large as possible, which could be achieved by settingmin_step
to some value greater than 0. If the chain was truly "burned in" atemi = 5000
, and the chain is 10000 elements long, then the returned acor timescale would ideally be only for iterations 5000 through 10000.I think a "smarter"
acor
could achieve this by sub-dividing the chain in powers of 2, trying allc
values up to a maximumc
, and then returning the acor time for the chain fragment for whichc
set to the largest value evaluated successfully. In some code I'm writing the logic currently looks like this:Thoughts on this proposal? As many of us acknowledge, very few chains are actually "burned in," but this would potentially yield of a way robustly determining when they are.