downsampled motion corrected movie

[saskiad]

Is your feature request related to a problem? Please describe. We used to have an avi of the downsampled (4Hz) motion corrected movie. This was extremely valuable for troubleshooting all sorts of issues related to motion correction, signal quality, image stability, etc. Not sure why this was removed, but not having it makes a lot of qc difficult.

Describe the solution you'd like I'd like an avi of the downsampled motion corrected movie

[DowntonCrabby]

@saskiad This issue relates to the lack of down sampled motion corrected movie in LIMS correct?

[DowntonCrabby]

@saskiad is the scope of this a specific project or is it related to all projects?

[saskiad]

@DowntonCrabby I would like this for any project I'm involved in (VIP SSN, V1 omFISH, SSF, etc). If other people don't find it helpful, I'm not going to dictate they use it

[DowntonCrabby]

The down-sampled motion corrected movies aren't well known files so I don't believe there was an obligation from the technology team to keep generating them or store them in a specific location. However, the full motion corrected movies are well known files so we should (in theory) be able to easily access those.

Is there any way to adapt the qc you used to do on the down sampled movies to work on the full movies? Or is the fact that they are down sampled the really integral part of how you did qc and manual review?

[saskiad]

The sampling rate really helps. full movies are really large and thus hard to open in a meaningful way. Regardless, there's no avi of the full movie either. I might not be clear on exactly what a well known file is, but the downsampled movie was being generated until recently, and there is still a dummy file with the same name that gets created - just without the movie. It's unclear to me how decisions are made on what there is and isn't an obligation to make.

[DowntonCrabby]

Can you provide the lims project codes for each of these projects? That would help me look up their structure & build lims queries

[saskiad]

omFISHGad2Pilot omFISHROBinjectionviruspilot omFISHcoregistrationpilot VIPSomaticV1Phase1

[matchings]

@saskiad are you able to view the .webm downsampled motion corrected movies that are in lims?

like this one, which is from a deepscope experiment: "\allen\programs\mindscope\production\omfish\prod0\specimen_1124090831\ophys_session_1131498234\ophys_experiment_1131799880\processed\1131799880_suite2p_motion_preview.webm"

[saskiad]

I am able to view them. They don't serve the needs that I have.

[saskiad]

is anything happening on this?

[DowntonCrabby]

I think we need more specific documentation of requirements to move forward. Some information that would help make a game plan:

• what is the ideal solution?

• In the absence of an ideal solution what is the minimum viable product? What do you absolutely need from these files/what purposes do they have that currently aren't met? Are there potential work arounds to make this work in the meantime?

• Are there restrictions or timelines we need to be aware of?

[matchings]

one consideration is availability of team Pika to do the work required to change the output to .avi instead of .webm (or add an .avi in addition). Tagging @wbwakeman

[saskiad]

• ideal solution is to return to the downsampled 4 Hz movies.
• I need a movie that has spatial/temporal downsampling. That is easy to open in, say FIJI, or play by itself. I want only the motion correct movie (not the two panels). I want to be able to easily scroll to different time points (if this is possible with webm, it is too slow to be useable).
• not having them is already slowing down work

[matchings]

also, @wbwakeman may know whether there was a reason to switch to .webm instead of .avi, perhaps a file size / storage issue? i dont recall why the switch was made

[matchings]

we do need to keep the 2 panel view for QC purposes though (ophys operators are using this regularly now i believe - maybe this is why we moved to webm?) so we should only add to, not replace, the current view.

[danielsf]

@saskiad Do you mind looking at the files here

/allen/aibs/informatics/danielsf/downsampling_example

I created some mp4s from an experiment whose motion correction I'm starting to doubt (it is a part of the prototype segmentation we've been working on for SSF). There are downsampled movies generated from legacy motion correction and Suite2P motion correction (you should be able to tell which is which from the file names). I used essentially the same spatial and temporal sampling routine we use to create the max projection images to create this movie. I can go into more detail if desired. I added red grid lines to help guide my eye. There are versions with and without grid lines.

My questions are:

1) Can you view these videos at all (we can generate AVIs; using mp4s is a little more consistent with our existing tooling, but not by much)?

2) Is this the kind of thing you need/want? If not, what needs to be changed, or am I barking up the wrong tree entirely?

3) There's clearly residual motion in this movie (more in the Suite2P version than the legacy version, but also in the legacy version). Is this an acceptable amount of residual motion, or does this count as a case where motion correction has failed?

[danielsf]

@saskiad @matchings Just tagging the above comment, since it came in right before the winter break and might have gotten missed.

[danielsf]

Pika has added a tool to our codebase to generate spatiotemporally downsampled TIFF files representing motion corrected movies

https://github.com/AllenInstitute/ophys_etl_pipelines/pull/433

Jun Zhuang has already verified that these artifacts fit his use case for validating motion correction.

Pika has not yet made these artifacts an automatic output of our motion correction pipeline. We know that is something we need to talk about. Before we do, I would like to ask (though I'm not sure who I should be asking or where): is there any reason to continue producing the .webm videos, or are those a dead end?

[DowntonCrabby]

@everythingevolves is to check this out to see if this solution works or to work with @danielsf to get the proper file format

[danielsf]

You can see examples of these files in the subdirectories of

/allen/programs/mindscope/workgroups/surround/motion_correction_labeling_2022

e.g.

./793618773/793618773_2Hz.tiff
./793618773/793618773_2Hz_with_reticle.tiff
./793618773/793618773_2Hz_side_by_side_with_reticle.tiff
./793618773/793618773_2Hz_side_by_side.tiff

(each directory corresponds to a different ophys_experiment_id associated with the SSF project)

[everythingevolves]

Yeah, the 2Hz_side_by_side_with_reticle version is what we need for this purpose. However, there is also a desire to have an easier/faster file format available (avi or webm version), because loading the tiffs when looking at many datasets is really cumbersome.

[danielsf]

@everythingevolves Are the current webms being produced by this pipeline sufficient, or do you want something about them changed?

[everythingevolves]

If all of the experiments will get the downsampled version of the webm file, it might be okay as is. [I believe in the past there was a non-downsampled version that took a very long time to load.]

[DowntonCrabby]

@everythingevolves is there an update on if the down sampled webm file is working for you? I'd like to get some clarity and closure on this ticket if possible.

[everythingevolves]

From my perspective, you can go ahead and close this ticket.

[danielsf]

I'm not sure this ticket should be closed yet. My interpretation of this conversation is that Pika needs to update the existing motion correction pipeline to automatically produces the 2Hz_side_by_side_with_reticle TIFFs in addition to the webm files.

I've opened this ticket

https://github.com/AllenInstitute/ophys_etl_pipelines/issues/483

to encapsulate that work.

[everythingevolves]

Perfect, thanks!