Question/ issue: Does mesmerise-core work with 3D movies?

[rajlakshmi-sawale]

Hi, does mesmerise-core work with 3D movies? I tried to run it with my xyzt data but the mcorr failed. Any help would be appreciated! Thanks!

[ethanbb]

Hi @rajlakshmi-sawale, is it failing at the local correlations step? If so, #300 is meant to address this, but it hasn't been merged yet.

[ethanbb]

OK I just merged that PR; can you try again after installing from the master branch? You will also need caiman version 1.11.2 or later.

[rajlakshmi-sawale]

Hi @ethanbb, Thanks for the quick reply! I'm not sure where it is failing.. I have attached what I see.. Sorry, I might not be running it in the right way in the first place as I am very new to this.

[ethanbb]

Yup, that looks normal, it ran into an error while running but it didn't tell you exactly what it is.

What does it show if you run print(df.iloc[-1, :].outputs['traceback'])?

[rajlakshmi-sawale]

This:

[rajlakshmi-sawale]

To run it on 3D data, do we have to specify any additional parameters? The only thing I changed so far in the provided notebook for 2D data was in this section:

# We will start with one version of parameters mcorr_params1 =\ { 'main': # this key is necessary for specifying that these are the "main" params for the algorithm { 'max_shifts': [5, 5, 2], 'strides': [24, 24, 6], 'overlaps': [12, 12, 2], 'max_deviation_rigid': 3, 'border_nan': 'copy', 'pw_rigid': True, 'gSig_filt': None }, }

[ethanbb]

Oh ok you might need to use a different index than -1. If you just print the df you should be able to see which row corresponds to the failed run ("outputs" should be a dict with "success": False) and then you can use that index.

[rajlakshmi-sawale]

[kushalkolar]

The traceback in the output doesn't have to be accessed manually, you can call get_output() and it will print the traceback nicely:

df.iloc[i].mcorr.get_output()

[ethanbb]

Ah, I think you also have to set is3D to true in your params['main'].

[rajlakshmi-sawale]

Hi, I re-installed it and got this error:

[kushalkolar]

Reload the dataframe from disk after running an item

[rajlakshmi-sawale]

[kushalkolar]

The message says you're out of RAM. What are the dimensions of the movie? How many pixels do you have per neuron/ROI? What's your temporal sampling rate? Can you post a mean projectiom of some of the planes.

[rajlakshmi-sawale]

It is 259,1024,28 with 1200 frames at 2 volumes per second. The neurons are approx 100-130 pixels.

[kushalkolar]

How much RAM do you have

On Tue, Aug 27, 2024, 7:27 PM Rajlakshmi Sawale @.***> wrote:

It is 259,1024,28 with 1200 frames at 2 volumes per second. The neurons are approx 100-130 pixels. MAX_avg.jpg (view on web) https://github.com/user-attachments/assets/e817ff44-50c5-497d-a6e6-93ef1017c9f6

— Reply to this email directly, view it on GitHub https://github.com/nel-lab/mesmerize-core/issues/312#issuecomment-2313136628, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACHXXRGSJWU6JVHNJBWXH2TZTSZJRAVCNFSM6AAAAABM552CDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJTGEZTMNRSHA . You are receiving this because you commented.Message ID: @.***>

[rajlakshmi-sawale]

128 GB

[kushalkolar]

Only thing I can think of is to reduce the number of processes, unless @ethanbb has any ideas. I've never done 3D mcorr or CNMF myself.

On Tue, Aug 27, 2024, 7:31 PM Rajlakshmi Sawale @.***> wrote:

128 GB

— Reply to this email directly, view it on GitHub https://github.com/nel-lab/mesmerize-core/issues/312#issuecomment-2313143612, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACHXXRGNVC43XTVMKZ5FYH3ZTSZW7AVCNFSM6AAAAABM552CDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJTGE2DGNRRGI . You are receiving this because you commented.Message ID: @.***>

[ethanbb]

I agree, reduce number of processes by setting the MESMERIZE_N_PROCESSES environment variable. That way it will not try to process as many chunks at a time and will use less memory.

Also, maybe you already did this, but in movies.py the function to3DFromPixelxTime should look like this:

def to3DFromPixelxTime(self, shape, order='F') -> 'movie':
    """
    Transform 2D movie (pixels x time) into 3D or 4D
    """
    return to_3D(self,shape[::-1],order=order).T

If not, you need to update your version of caiman.

[rajlakshmi-sawale]

Yes, the to3DFromPixelxTime function is as expected. After reducing the number of processes it gave this:

[ethanbb]

Oh interesting, I'm guessing I missed this because I only tried rigid 3d mcorr. Can take a look later today.

[ethanbb]

Yeah, sorry, the 3D mcorr PR had a bug which wasn't caught since I didn't add tests for both the rigid and piecewise cases.

Can you change line 115 of mesmerize_core/algorithms/mcorr.py from shifts += mc.z_shifts_els to shifts.append(mc.z_shifts_els) and see if it works now? If so I'll make a follow-up PR.

[rajlakshmi-sawale]

That worked! But now there seems to be some mismatch in dimensions for the visualisation..

[kushalkolar]

Thanks for hanging in there! Mesmerize-viz is pinned to a much older fastplotlib that's quite limited, we don't have the resources right now to update mesmerize-viz. You'll have to get the latest release of fastplotlib and use the ImageWidget to view your results, it's just a few lines rather than a one-liner that mesviz provides. ImageWidget examples are here: https://github.com/fastplotlib/fastplotlib/blob/main/examples/notebooks/image_widget.ipynb

[rajlakshmi-sawale]

Okay, thanks!

[rajlakshmi-sawale]

Hi, the visualization worked. Thanks for creating this amazing tool! I am now trying to run the CNMF but got this error..
Any suggestions?

[ethanbb]

Glad you're making progress! I believe this error is because you need to provide gSig as a 3-element list for 3D CNMF. The same is true for rf and stride in the patch group, if you're using patches. And finally you will have to disable the CNN by setting use_cnn in the quality group to False.

[rajlakshmi-sawale]

What about this?

[ethanbb]

I'm not sure, I haven't run into that one - you should try debugging it. It looks like at least one of the patch runs is giving an S of None when that's not expected. You can try to see why that's happening by setting a conditional breakpoint somewhere in CNMF.fit, for example. My guess would be that some of your patches are invalid in some way (empty?) and that tweaking your patch settings may fix it (but this is still a bug in caiman, probably).