Cannot run multiple successive iterations of mcorr?

[wmsheer]

I am trying to use mesmerize on a couple of miniscope recordings where I have lots of both rigid and non-rigid motion, so I'd like to be able to run multiple successive iterations of the mcorr algorithm to account for this. For example, I'd like to run one instance of the algorithm on my raw video where 'pw_rigid':True in the mcorr_params, take the output from that, and run a second instance where 'pw_rigid':False. So far, I've run into issues which has left me wondering whether I am doing something wrong or whether it's not currently possible to re-run an algorithm on a previous run's output.

So far, I've attempted this by doing the following:

1. Creating a batch item for the first algorithm run:

   df.caiman.add_item(
   algo='mcorr',
   input_movie_path=movie_path,
   params=mcorr_params1, #'pw_rigid':True
   item_name=movie_path.stem, 
   )

2. running the algorithm (successfully):

   
   row = df.iloc[0]

process = row.caiman.run() df = df.caiman.reload_from_disk()

3. adding a new batch item with new parameters, specifying the first mcorr iteration's output as the input movie (as per the example in the demo notebook for carrying over an mcorr output to a cnmf algorithm run):

df.caiman.add_item( algo='mcorr', input_movie_path=df.iloc[0], #using the first mcorr iteration's output params=mcorr_params2, #'pw_rigid':False item_name=df.iloc[0]['item_name'], )

As expected, `df` now has two rows. In this specific example, uuid for the first batch item (first algorithm run on raw movie) = 750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8 and for the second batch item (second algorithm run on output of first movie) = b0393392-e187-4a6b-ae23-87f1fafc2502

4. I then try to run the algorithm in the second batch item, this time unsuccessfully:

row = df.iloc[1] #corresponding to the output from the first iteration

process = row.caiman.run() df = df.caiman.reload_from_disk()

This has so far given me the following error, where the algorithm seems to not be able to decode the mmap filename before eventually giving up:

Running b0393392-e187-4a6b-ae23-87f1fafc2502 with local backend starting mc Decode mmap filename C:\miniscope_files\analyzed\5969\test\mesmerize-batch\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8-18_elsd1_600_d2_600_d3_1_order_F_frames_1000.mmap Decode mmap filename C:\miniscope_files\analyzed\5969\test\mesmerize-batch\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8-18_els__d1_600_d2_600_d3_1_order_F_frames_1000.mmap Decode mmap filename C:\miniscope_files\analyzed\5969\test\mesmerize-batch\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8-18_elsd1_600_d2_600_d3_1_order_F_frames_1000.mmap

WARNING:root:Movie average is negative. Removing 1st percentile. WARNING:root:Movie average is negative. Removing 1st percentile. WARNING:root:Movie average is negative. Removing 1st percentile.

Decode mmap filename C:\miniscope_files\analyzed\5969\test\mesmerize-batch\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8-18_els__d1_600_d2_600_d3_1_order_F_frames_1000.mmap mc failed, stored traceback in output

Per the traceback: 

The system cannot find the path specified: 'C:\miniscope_files\analyzed\5969\test\mesmerize-batch\b0393392-e187-4a6b-ae23-87f1fafc2502\750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8-18_elsd1_600_d2_600_d3_1_order_F_frames_1000_rigd1_600_d2_600_d3_1_order_F_frames_1000.mmap' -> 'C:\miniscope_files\analyzed\5969\test\mesmerize-batch\b0393392-e187-4a6b-ae23-87f1fafc2502\b0393392-e187-4a6b-ae23-87f1fafc2502-750a02ea-b0dc-4b8a-bdf1-fe46c216c1c8-18_elsd1_600_d2_600_d3_1_order_F_frames_1000_rigd1_600_d2_600_d3_1_order_F_frames_1000.mmap'

The path before the arrow (with both uuids) indeed exists in my directory, but the path after (with two of the second uuid) does not. I assumed I must have been passing the output of the first batch item as the input for the second run of the algorithm incorrectly until I tried running cnmf in a new batch item with the same input, which was successful:

df.caiman.add_item( algo='cnmf', # algo is cnmf input_movie_path=df.iloc[0], #using the first mcorr iteration's output params=params_cnmf, item_name=df.iloc[0]["item_name"], # use the same item name )

row = df.iloc[2] #batch item created immediately above

process = row.caiman.run() df = df.caiman.reload_from_disk()

I can't figure out whether I'm somehow pointing the mcorr algorithm to a nonexistent input for the second batch item but somehow avoiding this issue in my third batch item w/ the cnmf algorithm instead. Do you know what may be going on, here?

Thanks,
Will

[kushalkolar]

I wonder if this has something to do with how caiman stores information about the memmap file within the filename. What if you copy the mcorr memmap to somewhere in your input directory and rename it to something else, and use that file as the input?

By the way, when you set pw_rigid=True it does perform rigid motion correction (entire FOV), followed by piece-wise (elastic, i.e. non-rigid). You can perform multiple iterations of rigid (entire FOV) in a single algorithm-run by setting the niter_rig param. If you are performing elastic correction followed by entire FOV correction that seems bizarre and something else must be off. You usually correct for FOV shifts, and then piece-wise/elastic shifts within the FOV.

[wmsheer]

Oh great, thanks for pointing out niter_rig - totally overlooked that.

Anyways, I gave your suggestion a shot, but caiman didn't seem to like the memmap input (or it didn't like how I provided it, at least):

Running 59a14368-99f6-4065-990f-9a74fd1d6862 with local backend
starting mc
Decode mmap filename C:\miniscope_files\analyzed\5969\test\1mcorr_mmap.mmap
mc failed, stored traceback in output

From traceback:

ValueError: invalid literal for int() with base 10: 'mmap'

I've been looking around the caiman documentation and all the various youtube videos from workshops that I can find for how to best extract video files from mmaps (or at least extract other kinds of files that caiman's algorithms can make sense of) but I can't find a clear answer yet. Should I be providing something specific from the mmap file as the input video file? E.g., using the Yr variable from caiman.mmapping.load_memmap() doesn't seem to work.

[kushalkolar]

Sorry, by rename I should've specified that you want to keep the name after the uuid.

You'll need to keep this part because caiman encodes the movie dims within it:

-18_els__d1_600_d2_600_d3_1_order_F_frames_1000.mmap

Memmaps behave just like numpy arrays so you can do pretty much anything with them that you can with a numpy array. However for this purpose let's try and see where the issue lies, it should be possible to run mcorr directly from a memmap input.

However you may want to see if increasing your iterations of rigid correction works for you first. Are you cropping out the large black area which is common in miniscope recordings? I think it's better to do mcorr with just the cropped portion because it creates better templates.

[wmsheer]

Yep, increasing the iteration number in the mcorr params worked great for ironing out the remaining artifact. I am wary of cropping out anything yet because I haven't yet tried to cross-register caiman-processed videos across sessions so for now I'd prefer to manipulate the FOV as little as possible before doing so (whether this ends up being relevant or not). Plus other groups have suggested simply increasing the number of runs through normcorre to us before.

Out of curiosity, I went ahead and renamed the file again but left in that suffix - I was indeed able to run both mcorr again as well as CNMFE on this renamed memmap input. I was hoping this would also magically resolve a different error I was having, but I'll open an new issue thread for that. Thanks!

[kushalkolar]

With cropping what I mean is that if your entire FOV looks like this you could crop out a rectangle that only includes the "visible" FOV in green:

If you keep the same crop coordinates across sessions that should be fine? A lot of dark regions with no structure (i.e. the regions outside the visible FOV) can throw off the motion correction.

Thanks for trying to rename, I guess that caiman didn't like the uuid + memmap info filename for some reason because of how it uses hyphens and underscores to parse it out.

[wmsheer]

Yeah, that's what I was picturing in my head, but my FOVs typically have some cells near at least an edge or two of the frame. But I appreciate the suggestion, I'll definitely think about implementing in the future if simply bumping up the # of rigid iterations doesn't take care of it.

[kushalkolar]

closing due to inactivity, welcome to reopen :)