flatironinstitute / CaImAn

Computational toolbox for large scale Calcium Imaging Analysis, including movie handling, motion correction, source extraction, spike deconvolution and result visualization.
https://caiman.readthedocs.io
GNU General Public License v2.0
640 stars 370 forks source link

"v cannot be empty"; error related convolution in Numpy? #1276

Closed masa6161 closed 9 months ago

masa6161 commented 9 months ago

Your setup:

  1. Operating System (Linux, MacOS, Windows): Windows 10 Pro
  2. Hardware type (x86, ARM..) and RAM: CPU: AMD Ryzen Thread ripper 3970X 32 core RAM: 256GB
  3. Python Version (e.g. 3.9): 3.10.8 and spyder IDE 5.5.1
  4. Caiman version (e.g. 1.9.12): 1.10.2
  5. Which demo exhibits the problem (if applicable): Modified script based on demo_caiman_basic.py and demo_pipeline.ipynb
  6. How you installed Caiman (pure conda, conda + compile, colab, ..): Followed official instructions (conda + manba with conda-forge repo. )
  7. Details: This error occurred in the initial estimation stage (especially greadyROI()) whether using "multiprocessing"(cluster number was tested from 4 to 16) or not. Demo files ("Sue_2x_300040-46.tif" and "Mesoscope") were correctly processed, but my data (512x512x36000, 5 separated tiffs) were not. My data were motion-corrected, so this process was skipped, and then memory-mapped files were directly created in the script. Does the problem follow from the total size of data (17.5GB total of my data vs. <1GB of demo data)?

My parameter set in the signal extraction is below; fr = 30 decay_time = 0.4
p = 2
gnb = 2
merge_thr = 0.8
rf = 64
stride_cnmf = 12
K = 30
gSig = np.array([4, 4])
gSiz = 2gSig+1
method_init = 'greedy_roi'
ssub = 1
tsub = 1
rolling_length_fac = 3
fr; bas_nonneg = True

And tracebacks are here;

File z:\pythoncode\roidetection\caiman_240210.py:227 in do_CNMF cnm = cnm.fit(images)

File ~\anaconda3\envs\caiman\lib\site-packages\caiman\source_extraction\cnmf\cnmf.py:493 in fit self.initialize(Y)

File ~\anaconda3\envs\caiman\lib\site-packages\caiman\source_extraction\cnmf\cnmf.py:963 in initialize initialize_components(Y, sn=estim.sn, options_total=self.params.to_dict(),

File ~\anaconda3\envs\caiman\lib\site-packages\caiman\source_extraction\cnmf\initialization.py:335 in initializecomponents Ain, Cin, , b_in, f_in = greedyROI(

File ~\anaconda3\envs\caiman\lib\site-packages\caiman\source_extraction\cnmf\initialization.py:806 in greedyROI rho_s = scipy.signal.lfilter(rolling_filter, 1., rho**2)

File ~\anaconda3\envs\caiman\lib\site-packages\scipy\signal_signaltools.py:2138 in lfilter out_full = np.apply_along_axis(lambda y: np.convolve(b, y), axis, x)

File ~\anaconda3\envs\caiman\lib\site-packages\numpy\lib\shape_base.py:379 in apply_along_axis res = asanyarray(func1d(inarr_view[ind0], *args, **kwargs))

File ~\anaconda3\envs\caiman\lib\site-packages\scipy\signal_signaltools.py:2138 in out_full = np.apply_along_axis(lambda y: np.convolve(b, y), axis, x)

File ~\anaconda3\envs\caiman\lib\site-packages\numpy\core\numeric.py:833 in convolve raise ValueError('v cannot be empty')

ValueError: v cannot be empty

masa6161 commented 9 months ago

After posting this issue, I've reviewed and rewrote all of my (modified) scripts. As a result, using 16 clusters, the initial estimation step was passed, and now, the refinement step is starting.
Perhaps, by my code fault (the part of a programatical determination of some parameters, frame-rate, rf, cnmf_stride, gSig, etc., from metadata), this issue happened. If so, I apologize for the premature submission. If this problem occurs after some runs, I'd like to inform you again.

EricThomson commented 9 months ago

Ok thanks for the follow up. Closing the issue for now, but please reopen if this reproduces