Getting denoised dF/F

[kingmiaomiao]

Hi, I'm trying to understand the df/f calculation, and when (or if) residuals are used (this is related to the message I posted on Gitter). When I call df.iloc[1].cnmf.run_detrend_dfof(use_residuals = False), mesmerize then calls estimates.detrend_df_f, which eventually goes to detrend_df_f in CaImAn/caiman/source_extraction/cnmf/utilities.py. However, before that, the code in estimates.py seem to ignore the flag use_residuals = False:

        if self.C is None or self.C.shape[0] == 0:
            logging.warning("There are no components for DF/F extraction!")
            return self

        if use_residuals:
            if self.R is None:
                if self.YrA is None:
                    R = None
                else:
                    R = self.YrA
            else:
                R = self.R
        else:
            R = None

        self.F_dff = detrend_df_f(self.A, self.b, self.C, self.f, self.YrA,
                                  quantileMin=quantileMin,
                                  frames_window=frames_window,
                                  flag_auto=flag_auto, use_fast=use_fast,
                                  detrend_only=detrend_only)
        return self

Could you explain what happens there, and how to get a denoised df/f, without the residuals? What should I change to my code below?

df.iloc[1].cnmf.run_detrend_dfof(use_residuals = False)
cnmf_data = df.iloc[1].cnmf.get_output()
F_dff = cnmf_data.estimates.F_dff

Right now the df/f does not seem to be denoised.

Thanks in advance for your help!

[kushalkolar]

this is more of a caiman question than mesmerize question, @EricThomson might have an answer

[EricThomson]

Yes there is a use_residual keyword that currently has no effect when calculating dff. See https://github.com/flatironinstitute/CaImAn/issues/1188

[vncntprvst]

Thanks @EricThomson for pointing to that open issue, and working on solving this bug :) I'll comment on the CaImAn repo's thread for that issue.

[kingmiaomiao]

Thank you! In the meantime, can I just change this one line in CaImAn, when estimates calls detrend_df_f, replacing self.YrA with R? I will update caiman when you push the fixed code. I assume you will create a new release?

[EricThomson]

That's the fix :smile: it's currently in a PR that should be in the next release.

[kingmiaomiao]

It works!! Thank you!!!