AllenInstitute / ophys_etl_pipelines

Pipelines and modules for processing optical physiology data
Other
9 stars 5 forks source link

neuropil correction coefficient larger than 1 #538

Closed jkim0731 closed 1 year ago

jkim0731 commented 2 years ago

We found that the neuropil correction coefficient (r) has values larger than 1, sometimes larger than 2. These were observed in 'VisualBehavior4areax2depth' projects, but could be in other projects as well.

We think this is related to #512. Having large r value led to very high dff amplitude.

When r value is estimated as greater than 1, it should be replaced by the mean of other r values less than 1 in the same experiment. The r value search should be limited to [0,2], so having r value > 2 suggests this limit was not applied as intended.



Here are some example traces with r > 1 (r value noted in red in the top plot; bottom row shows dff)

image

image

image

jkim0731 commented 1 year ago

Addendum: when there are less than 5 ROIs with r within (0,1), then r = 0.7

mikejhuang commented 1 year ago

Hi @jkim0731

The requested fix to fill in the R vals > 1 has been completed and merged (including the addendum).

Note that the R values can be > 2 since r_range = [0 2] is only the initial search range. There are conditionals in the fitting algorithm that expands this range if the estimated loss is at a minimum at the boundary.

jkim0731 commented 1 year ago

Thank you, @mikejhuang!

One question about r > 2 - Are those r values > 2 replaced by the mean of r's less than one at the end?

mikejhuang commented 1 year ago

@jkim0731 Yep! All r > 1 gets filled in by the mean of rois with r <= 1 if the count(rois <= 1) >= 5, else it's filled with 0.7. I don't understand why the search range goes beyond 1 if it's going to get filled in.

mikejhuang commented 1 year ago

Also, the neuropil corrected trace is recalculated with the new r value and the RMSE is recalculated with the new neuropil corrected trace.