Replace regfilt with Python code

[tsalo]

Summary

This is part of #2. I'm pretty sure we can use numpy.linalg.lstsq to handle this regression. I just have some questions about non-aggressive vs. aggressive denoising.

Aggressive denoising

Regress bad components out of data and retain residuals as the denoised data. That's it, right?

Non-aggressive denoising

Regress good components out of bad components and retain residuals, then regress updated bad components (i.e., the residuals) out of data and retain those residuals as the denoised data. Is that correct?

[eurunuela]

I'd say that sounds right. What do you think @CesarCaballeroGaudes ?

[CesarCaballeroGaudes]

Not exactly.... the aggressive one is correct (i.e. including only the bad components and retain residuals). However, the non-aggressive denoising is as follows: 1- Solve the least squares regression with all of the components (bad and good). 2- Compute the fit of the bad components (i.e. bad IC least squares coefficients * bad IC time series) and then subtract from the signal to obtain the residuals. The approach you are proposing is what we labelled as the conservative approach in the following paper by @smoia (https://www.biorxiv.org/content/10.1101/2020.08.18.256479v1) Therefore, it is simpler and it only performs one regression.

[tsalo]

Thanks for the clarification! Do you think the conservative approach is worth incorporating down the line?

[CesarCaballeroGaudes]

Not yet!! The point is that then you are biasing your results towards the good components, which get all the variance shared with the bad ones. Is that good? Strict statisticians might say "NO, it is not appropriate". I would say "It depends on the circumstances because in some scenarios the bad regressors might be completely killing your effect of interest". So my solution is the non-agressive approach (or moderate as we named in the paper)