search

MouseLand / suite2p

cell detection in calcium imaging recordings
http://www.suite2p.org
GNU General Public License v3.0
344 stars 240 forks source link

BUG: Running suite2p interrupted (no file/directory found) #973

Closed marytunun closed 1 year ago

marytunun commented 1 year ago

Describe the issue:

I opened an issue here before about installation: https://github.com/MouseLand/suite2p/issues/946 Tried the suggested solution, but the sample data still had a 'strip data' issue, so I tried on tiff images from my actual lab. I have now a folder with those tiff images for analysis, all of them are 256x256.

Running suite2p almost worked, seems that it successfully went through steps of converting to binary, registration, ROI detection, extraction, classification and spike convolution. BUT after that I have errors, and it seems like those errors are not because of something I did..

Is suite2p for some reason not able to create thosee specific files/directories? I require your assistance with this error, thank you.

Reproduce the code example:

{'data_path': ['C:/Users/Maria/Documents/suite2p_docs/lab_data_samples/Ca_imaging_organoids/256tiffs'], 'subfolders': [], 'save_path0': 'C:/Users/Maria/Documents/suite2p_docs/save_path', 'fast_disk': 'C:/Users/Maria/Documents/suite2p_docs/fast_disk', 'input_format': 'tif'}
tif
** Found 6330 tifs - converting to binary **
2000 frames of binary, time 139.29 sec.
4000 frames of binary, time 274.81 sec.
6000 frames of binary, time 406.76 sec.
time 429.00 sec. Wrote 6330 frames per binary for 2 planes
>>>>>>>>>>>>>>>>>>>>> PLANE 0 <<<<<<<<<<<<<<<<<<<<<<
NOTE: not registered / registration forced with ops['do_registration']>1
      (no previous offsets to delete)
NOTE: applying default C:\Users\Maria\.suite2p\classifiers\classifier_user.npy
----------- REGISTRATION
Reference frame, 4.09 sec.
Registered 500/6330 in 11.82s
Registered 1000/6330 in 17.65s
Registered 1500/6330 in 23.84s
Registered 2000/6330 in 29.45s
Registered 2500/6330 in 33.80s
Registered 3000/6330 in 38.73s
Registered 3500/6330 in 43.34s
Registered 4000/6330 in 48.18s
Registered 4500/6330 in 53.37s
Registered 5000/6330 in 57.28s
Registered 5500/6330 in 61.32s
Registered 6000/6330 in 65.27s
Registered 6330/6330 in 67.96s
----------- Total 73.90 sec
Registration metrics, 12.05 sec.
----------- ROI DETECTION
Binning movie in chunks of length 10
Binned movie of size [633,208,208] created in 1.10 sec.
NOTE: estimated spatial scale ~12 pixels, time epochs 1.00, threshold 10.00 
0 ROIs, score=75.32
Detected 13 ROIs, 6.14 sec
After removing overlaps, 11 ROIs remain
----------- Total 7.46 sec.
----------- EXTRACTION
Masks created, 60.64 sec.
Extracted fluorescence from 11 ROIs in 6330 frames, 11.77 sec.
----------- Total 72.49 sec.
----------- CLASSIFICATION
['skew', 'npix_norm', 'compact']
----------- SPIKE DECONVOLUTION
----------- Total 0.01 sec.
Plane 0 processed in 165.99 sec (can open in GUI).
>>>ERROR<<<
Traceback (most recent call last):
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 194, in _run_module_as_main
>>>ERROR<<<
    return _run_code(code, main_globals, None,
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\__main__.py", line 81, in <module>
>>>ERROR<<<
    main()
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\__main__.py", line 74, in main
>>>ERROR<<<
    run_s2p(ops, db)
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 506, in run_s2p
>>>ERROR<<<
    op = np.load(ops_path, allow_pickle=True).item()
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\numpy\lib\npyio.py", line 405, in load
>>>ERROR<<<
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/Maria/Documents/suite2p_docs/save_path\\suite2p\\plane1\\ops.npy'
Interrupted by error (not finished)

Error message:

No response

Version information:

suite2p v0.12.2.dev30+g828b5e2

Context for the issue:

Cannot open my tiff images because the running is interrupted

carsen-stringer commented 1 year ago

Perhaps you didn't create the folder you're trying to save in?

generalciao commented 1 year ago

Do you have / are you expecting 2 planes? I.e. do you have a fast focus switching setup (ETL, piezo)? Because the suite2p output implies it thinks your data contains 2 planes, and the error occurs for a 2nd plane folder.

Note at the beginning the following output: Wrote 6330 frames per binary for 2 planes

And at the end the FileNotFoundError refers to "plane1", which would be the second plane: 'C:/Users/Maria/Documents/suite2p_docs/save_path\\suite2p\\plane1\\ops.npy'

I have no personal experience with multiplane analysis with suite2p, so this might still be an installation bug as you suspect, but it might also be a mismatch between your data and your settings?

marytunun commented 1 year ago

Do you have / are you expecting 2 planes? I.e. do you have a fast focus switching setup (ETL, piezo)? Because the suite2p output implies it thinks your data contains 2 planes, and the error occurs for a 2nd plane folder.

Note at the beginning the following output: Wrote 6330 frames per binary for 2 planes

And at the end the FileNotFoundError refers to "plane1", which would be the second plane: 'C:/Users/Maria/Documents/suite2p_docs/save_path\\suite2p\\plane1\\ops.npy'

I have no personal experience with multiplane analysis with suite2p, so this might still be an installation bug as you suspect, but it might also be a mismatch between your data and your settings?

Actually, my data has 1 plane, and it is also 1 in the 'main settings'. Do you think I should put 0 planes instead? Although that would be really weird...

marytunun commented 1 year ago

Perhaps you didn't create the folder you're trying to save in?

The folder exists for sure. But what I also tried now is to not specify separate folders for save_path and fast_disk, and I got another error that started already during the extraction process:

----------- EXTRACTION ---ERROR--- Traceback (most recent call last): File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 194, in _run_module_as_main ---ERROR--- return _run_code(code, main_globals, None, File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 87, in _run_code ---ERROR--- exec(code, run_globals) File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p__main.py", line 81, in ---ERROR--- main() File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\main__.py", line 74, in main run_s2p(ops, db) File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 518, in run_s2p ---ERROR--- op = run_plane(op, ops_path=ops_path) File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 333, in run_plane ops = pipeline(f_reg, f_raw, f_reg_chan2, f_raw_chan2, run_registration, ops, File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 170, in pipeline ---ERROR--- stat, F, Fneu, F_chan2, Fneu_chan2 = extraction.extraction_wrapper( File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\extraction\extract.py", line 189, in extraction_wrapper ---ERROR--- cell_masks, neuropil_masks0 = create_masks(stat, Ly, Lx, ops) File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\extraction\masks.py", line 13, in create_masks ---ERROR--- cell_pix = create_cell_pix(stats, Ly=Ly, Lx=Lx, File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\extraction\masks.py", line 50, in create_cell_pix filt = percentile_filter(lammap, percentile=lam_percentile, File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\scipy\ndimage_filters.py", line 1467, in percentile_filter ---ERROR--- return _rank_filter(input, percentile, size, footprint, output, mode, File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\scipy\ndimage_filters.py", line 1334, in _rank_filter _nd_image.rank_filter(input, rank, footprint, output, mode, cval, ---ERROR--- MemoryError Interrupted by error (not finished)

marytunun commented 1 year ago

Didn't really solve the issue, but I just jumped to my lab's data instead of the sample data and got other errors, which I better describe in a separate thread