search

flatironinstitute / bayes-kit

Bayesian inference and posterior analysis for Python
MIT License
43 stars 3 forks source link

autocorr_fft - truncating to be a power of 2? #11

Closed magland closed 1 year ago

magland commented 1 year ago

https://github.com/flatironinstitute/bayes-kit/blob/8f8c494cf65ce39f0982566b54ed1a9a277f8c82/bayes_kit/ess.py#L8-L26

Seems that you are truncating the Fourier coefficients to force them to be a power of 2. When transforming back the power spectrum, this will have the effect of sampling on a different grid than the original data. This can be solved by omitting the size parameter, and just doing the full fft. You may also consider keeping only the first half of the output array.

bob-carpenter commented 1 year ago

Thanks! That's an easy change, and I'll try to follow up with you so that I can understand what's going on. It looked like this was getting the same autocorrelation estimates as the built-in in Python, but maybe I wasn't testing enough conditions.

And yes, we should probably only keep the first half of the output as each lag is estimated using one fewer data points than the last, and the whole thing's stochastic, so the noise piles up quickly.

magland commented 1 year ago

Thanks! That's an easy change, and I'll try to follow up with you so that I can understand what's going on. It looked like this was getting the same autocorrelation estimates as the built-in in Python, but maybe I wasn't testing enough conditions.

Strange. Did you try it for only power-of-2 sized examples, perhaps?

magland commented 1 year ago

I agree now that the fft code is correct. Closing.