2p event detection follow-up

[djkapner]

This is a follow-up to #75

Tasks:

• [x] provide science team with event detection results of scientifica data sub-sampled by 3. Do so over a range of noise multiplier values.
• [x] incorporate science team feedback into default values of module.
• [ ] tentative: remove downsample/upsample logic from production event detection code.

Validation:

• [x] science team feedback received.
• [ ] code modified and PR merged.

[djkapner]

Example plots showing the sub-sampling effect:

Original (31Hz) compared to sub-sampled ( / 3) with noise_multipler = 1.0:

Original (31Hz) compared to sub-sampled ( / 3) with noise_multipler = 1.5:

Original (31Hz) compared to sub-sampled ( / 3) with noise_multipler = 2.0:

etc.

These plots are from this notebook: /allen/aibs/informatics/danielk/event_detection/event_detection_comparison.ipynb which has been updated (in a quick and dirty way)to handle this downsampled business.

so far, I have done noise_multipliers 1.0, 1.5, 2.0, 2.5, 3.0 Peter asked for steps of 0.1, so I will fill that in. results are available for any of the 42 scientifica experiments.

@matchings @ledochowitsch

[matchings]

@djkapner is it possible to adapt the notebook to plot the events for different noise_multipliers overlaid on top of each other in the interactive plot? Perhaps just for a subset of the values, such as 1, 2, 3, plus the dFF trace. I think this would really help to be able to compare them directly against each other and see which yields undesirable amounts of false positives / negatives compared to the dFF trace. Its hard to judge which noise_multiplier is best when viewing them in different figures.

[djkapner]

@matchings It is possible. I'd like to turn this back over to the science team so I can take a crack at the motion correction problems.

[djkapner]

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.0
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.1
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.2
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.3
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.4
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.5
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.6
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.7
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.8
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.9
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.0
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.1
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.2
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.3
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.4
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.5
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.6
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.7
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.8
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.9
/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_3.0

are complete

[ledochowitsch]

Thank you so much for providing these – I have to finish a bit more of a systematic analysis, but if I am interpreting these results correctly, the appropriate multiplier compensating for the downsampling effect is < 2.

-Peter

From: Dan Kapner notifications@github.com Date: Tuesday, January 19, 2021 at 12:52 PM To: AllenInstitute/ophys_etl_pipelines ophys_etl_pipelines@noreply.github.com Cc: Peter Ledochowitsch peterl@alleninstitute.org, Mention mention@noreply.github.com Subject: Re: [AllenInstitute/ophys_etl_pipelines] 2p event detection follow-up (#147)

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.0

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.1

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.2

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.3

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.4

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.5

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.6

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.7

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.8

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_1.9

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.0

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.1

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.2

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.3

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.4

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_2.5

/allen/aibs/informatics/danielk/event_detection/output/subsampled/multiplier_3.0

are complete

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAllenInstitute%2Fophys_etl_pipelines%2Fissues%2F147%23issuecomment-763132199&data=04%7C01%7C%7C85631db80fdd4308a8b608d8bcbc13b1%7C32669cd6737f4b398bddd6951120d3fc%7C0%7C0%7C637466863267929906%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KsZZVCwF7v4UYkTQ4FRtVbzGv2T5gN%2FZpL0kEw56MSM%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABVLHVXJRX3VXJL2H5MXDK3S2XWHHANCNFSM4WJLPJEA&data=04%7C01%7C%7C85631db80fdd4308a8b608d8bcbc13b1%7C32669cd6737f4b398bddd6951120d3fc%7C0%7C0%7C637466863267939896%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1EO8Dc4PbeDZQ25rHbEm6mk7Kql%2FyQQnWn%2FQtSbPOVw%3D&reserved=0.

[matchings]

@djkapner good point about switching to motion correction. i will try to make the plot i am imagining later today.