Correct Usage of Parameters

[HunterTom94]

For better support, please use the template below to submit your issue. When your issue gets resolved please remember to close it.

Sometimes errors while running CNMF occur during parallel processing which prevents the log to provide a meaningful error message. Please reproduce your error with setting dview=None.

If you need to upgrade CaImAn follow the instructions given in the wiki.

• Tell us about your operating system (Linux/macOS/Windows), Python version (3.x), and working environment (Spyder/Jupyter Notebook/other) I am using Ubuntu 18, python 3.6 on JupyterNotebook.

• Describe the issue that you are experiencing When running modified version of demo_pipeline.ipynb using my own video, I have index error when

1. running the cell that show up accepted components
2. running the cell that show final results

There is no error showing the rejected components.

I am wondering if this error is caused by I putting in the wrong number when setting up the parameter

K = 4 # number of components per patch

Thank you very much!

• Copy error log below Error when showing accepted components: IndexError Traceback (most recent call last) in 1 # accepted components ----> 2 cnm2.estimates.nb_view_components(img=Cn, idx=cnm2.estimates.idx_components)

~/anaconda3/envs/caiman/lib/python3.6/site-packages/caiman/source_extraction/cnmf/estimates.py in nb_view_components(self, Yr, img, idx, denoised_color, cmap, thr) 345 self.C[idx], self.b, self.f, 346 self.dims[0], self.dims[1], YrA=self.R[idx], image_neurons=img, --> 347 thr=thr, denoised_color=denoised_color, cmap=cmap) 348 return self 349

~/anaconda3/envs/caiman/lib/python3.6/site-packages/caiman/utils/visualization.py in nb_view_patches(Yr, A, C, b, f, d1, d2, YrA, image_neurons, thr, denoised_color, cmap) 183 cc1 = [cor['coordinates'][:, 0] for cor in coors] 184 cc2 = [cor['coordinates'][:, 1] for cor in coors] --> 185 c1 = cc1[0] 186 c2 = cc2[0] 187

IndexError: list index out of range

Error when showing final results: IndexError Traceback (most recent call last)

in ----> 1 cnm2.estimates.nb_view_components(img=Cn, denoised_color='red') 2 print('you may need to change the data rate to generate this one: use jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10 before opening jupyter notebook') ~/anaconda3/envs/caiman/lib/python3.6/site-packages/caiman/source_extraction/cnmf/estimates.py in nb_view_components(self, Yr, img, idx, denoised_color, cmap, thr) 340 caiman.utils.visualization.nb_view_patches(Yr, self.A, self.C, 341 self.b, self.f, self.dims[0], self.dims[1], YrA=self.R, image_neurons=img, --> 342 thr=thr, denoised_color=denoised_color, cmap=cmap) 343 else: 344 caiman.utils.visualization.nb_view_patches(Yr, self.A.tocsc()[:,idx], ~/anaconda3/envs/caiman/lib/python3.6/site-packages/caiman/utils/visualization.py in nb_view_patches(Yr, A, C, b, f, d1, d2, YrA, image_neurons, thr, denoised_color, cmap) 183 cc1 = [cor['coordinates'][:, 0] for cor in coors] 184 cc2 = [cor['coordinates'][:, 1] for cor in coors] --> 185 c1 = cc1[0] 186 c2 = cc2[0] 187 IndexError: list index out of range - **If you're not reporting an error, type your message below**

[HunterTom94]

Parameters used to generate this graph is 'rf = 15' and 'K = 4' , 'gSig = [4, 4]'

My guess is that there is no components got accepted. In the picture attached, it seems that all neurons are not in any component found. I am not sure which direction I shall tune the parameters to.

Thank you very much!

[HunterTom94]

My understanding of patches is that it helps with the parallel processing. Will the number of patches (determined by two lines below) influence the result?

rf # half-size of the patches in pixels. e.g., if rf=25, patches are 50x50 stride_cnmf # amount of overlap between the patches in pixels

[epnev]

@HunterTom94 I see that you closed the issue. Did you figure out what was going wrong?

[HunterTom94]

@epnev I believe the index error is caused by the fact that there is no accepted components.

However, I still have the question about choosing the right size of patches and the corresponding k (number of components in a patch) values.

Neurons are not evenly distributed in FOV. Thus, some patches may have more neurons than the others. In that case, what should k be?

Also, in line 106 of cnmf.py it says k is number of neurons in a patch. However, in demo_pipeline.ipynb, it says k is the number of component in a patch. I thought they are different. Neuron should be accepted component; component should be all candidate to be accepted or rejected.

Therefore, I am wondering that k really should be and that if different rf value will lead to different results. Currently, I think the essential parameters in evaluation should be rf, k and gSig, am I correct?

Thank you very much!

[epnev]

@HunterTom94 We do not distinguish between neurons and in components in the comments/documentation. In general k should reflect the maximum number of neurons you see in a patch. For patches with less neurons, false positives will be introduced and the component evaluation step will attempt to remove them.

[HunterTom94]

@epnev Thank you for your reply. However, this still does not explain why in the screenshot I provided, in the top region and bottom region (backgrounds), found components are checker-board shaped, while in the middle where neuron actually lies. No a single neuron get identified as a component. In fact, all neurons seem to be "perfectly avoided" being detected as a component. Is there a way to fix this? There might be something wrong with my parameter settings but I do not know where it was wrong.

I used the following paramters for this 170*170 video:

dataset dependent parameters

fr = 30 # imaging rate in frames per second decay_time = 0.4 # length of a typical transient in seconds

motion correction parameters

strides = (48, 48) # start a new patch for pw-rigid motion correction every x pixels overlaps = (24, 24) # overlap between pathes (size of patch strides+overlaps) max_shifts = (6,6) # maximum allowed rigid shifts (in pixels) max_deviation_rigid = 3 # maximum shifts deviation allowed for patch with respect to rigid shifts pw_rigid = True # flag for performing non-rigid motion correction

parameters for source extraction and deconvolution

p = 1 # order of the autoregressive system gnb = 2 # number of global background components merge_thresh = 0.8 # merging threshold, max correlation allowed rf = 15 # half-size of the patches in pixels. e.g., if rf=25, patches are 50x50 stride_cnmf = 6 # amount of overlap between the patches in pixels K = 4 # number of components per patch gSig = [1, 1] # expected half size of neurons in pixels method_init = 'greedy_roi' # initialization method (if analyzing dendritic data using 'sparse_nmf') ssub = 1 # spatial subsampling during initialization tsub = 1 # temporal subsampling during intialization

parameters for component evaluation

min_SNR = 2.0 # signal to noise ratio for accepting a component rval_thr = 0.85 # space correlation threshold for accepting a component cnn_thr = 0.99 # threshold for CNN based classifier cnn_lowest = 0.1 # neurons with cnn probability lower than this value are rejected

[epnev]

@HunterTom94 There is something weird with the figure you posted since the background appears to have higher values than the places were neurons are visible. If you send me the file and the script you're using I can try to replicate.

[HunterTom94]

@epnev Thank you for your reply! The video file is in this link: https://drive.google.com/open?id=1Q6hJj0uisfTvem3rRQLYk1qtxZnFMWYc

The script file is in this link: https://drive.google.com/open?id=1uczEY5U3SLt6UmC_GQuE85oOs1qsCPe8

Thank you very much again for troubleshooting with me!

[epnev]

@HunterTom94 I took a look at the dataset. There is a couple of things:

• I cannot reproduce the figure you're sending me with your script. When I run your script I do not get the tiling of components on the upper part.
• The dataset is very short with barely any activity shown. If I plot the average value of each frame I get the following trace: This shows that there is barely any activity (also observed by watching the movie). The algorithm works by identifying positive transients around a baseline that correspond to neural activity transients. All variation seems to be due a slowly decreasing baseline.
• Increasing the gSig value identifies more components
• I think you should try on longer datasets (more than 1000 frames) and if you can obtain more data you should consider increasing the spatial resolution to make the components more visible.

[HunterTom94]

@epnev Thank you for your explanations! Also, I am wondering what are the difference between black circles and red contours displayed in the components viewing graph. They seems to be independent of each other, although sometimes I see black circles lie in the center of the red contoured area.