Related to #1081. Rerunning suite2p on multiplane data where previously ignore_flyback=2 was set gives rise to an error that I believe occurs only because it is not taken into account that plane2 has to be ignored again.
Reproduce the code example:
The easiest way to reproduce is to run suite2p on multiplane data and set ignore_flyback>1 and combined=1. And simply run it another time. I run suite2p from the terminal.
import matplotlib.pyplot as plt
import suite2p
import argparse
from pathlib import Path
if __name__ == "__main__":
class ParseKwargs(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, dict())
for value in values:
key, value = value.split('=')
getattr(namespace, self.dest)[key] = value
parser = argparse.ArgumentParser(description="Run analysis script with datalad run")
parser.add_argument('inputs', nargs='*', help='-i path/to/2p/dataset/subject/*drug/* (-i ...) -o path/to/suite2p/dataset/subject/combined_exp')
parser.add_argument('outputs')
parser.add_argument('--settings', nargs='*', action=ParseKwargs)
args = parser.parse_args()
data_path = args.inputs
save_path0 = args.outputs
# set the suite2p settings
ops = suite2p.default_ops()
# --- main ---
# ops['nplanes'] = args.nplanes
ops['tau'] = 1.25 # timescale of gcamp to use for deconvolution, 0.2 for GCaMP8m from the paper
# ops['fs'] = 10 # sampling rate of recording, determines binning for cell detection, in Hz
# --- output ---
ops['reg_tif'] = True # save registered tifs for manual segmentation and fissa
ops['move_bin'] = False # make sure to ignor binary files and reg_tif to save time for datalad run cmd
# --- registration ---
ops['nimg_init'] = 600 # default 300, maybe higher helps with low signal-to-noise?
ops['batch_size'] = 900 # we will decrease the batch_size in case low RAM on computer
# --- ROI detection ---
ops['roidetect'] = True
ops['spatial_scale'] = 0 # 0=automatic, 1 (=6 pixels), 2 (=12 pixels), 3 (=24 pixels), or 4 (=48 pixels)
ops['connected'] = False # do not require ROIs to be fully connected as suggested for dendrites/boutons
ops['threshold_scaling'] = 1.8 # we are increasing the threshold for finding ROIs to limit the number of non-cell
# ROIs found (sometimes useful in gcamp injections)
ops['max_overlap'] = 1.0 # to throw out no ROIs
ops['max_iterations'] = 100
ops['sparse_mode'] = True # default
ops['denoise'] = 0 # if True, sparse_mode has to be True, too
# --- cellpose detection ---
ops['anatomical_only'] = 0
ops['diameter'] = 0
# --- neuropil ---
ops['allow_overlap'] = False
# integrate any further custom settings
if args.settings is not None:
for key, val in args.settings.items():
try:
ops[key] = eval(val)
except NameError:
ops[key] = val
if ops['nplanes'] == 3:
ops['ignore_flyback'] = [2]
ops['fs'] = 10
data_path.sort()
sub_exp = '/'.join(output_path.split('/')[-2:])
fast_disk = f'/home/meng/Fast_disk/{sub_exp}'
Path(fast_disk).mkdir(parents=True, exist_ok=True)
db = {'data_path': data_path, 'save_path0': save_path0, 'fast_disk': fast_disk}
output_ops = suite2p.run_s2p(ops=ops, db=db)
Error message:
{'data_path': ['01_data/01_raw_data/02_Psychedelics_L5b_2p/231113D_Sim-Cre/2023-12-19_Saline/Saline_0', '01_data/01_raw_data/02_Psychedelics_L5b_2p/231113D_Sim-Cre/2023-12-19_Saline/Saline_0.9', '01_data/01_raw_data/02_Psychedelics_L5b_2p/231113D_Sim-Cre/2023-12-20_LSD/LSD_0', '01_data/01_raw_data/02_Psychedelics_L5b_2p/231113D_Sim-Cre/2023-12-20_LSD/LSD_0.4'], 'save_path0': '01_data/02_preprocessed_data/02_Psychedelics_L5b_suite2p/231113D_Sim-Cre/Saline_LSD', 'fast_disk': '/home/meng/Fast_disk/231113D_Sim-Cre/Saline_LSD'}
tif
** Found 468 tifs - converting to binary **
3600 frames of binary, time 16.63 sec.
7200 frames of binary, time 33.88 sec.
10800 frames of binary, time 51.08 sec.
14400 frames of binary, time 69.57 sec.
18000 frames of binary, time 97.42 sec.
21600 frames of binary, time 125.81 sec.
25200 frames of binary, time 153.15 sec.
28800 frames of binary, time 180.92 sec.
32400 frames of binary, time 208.34 sec.
36000 frames of binary, time 234.75 sec.
39600 frames of binary, time 260.59 sec.
43200 frames of binary, time 287.77 sec.
46800 frames of binary, time 314.89 sec.
50400 frames of binary, time 343.38 sec.
54000 frames of binary, time 370.69 sec.
57600 frames of binary, time 398.88 sec.
61200 frames of binary, time 426.36 sec.
64800 frames of binary, time 456.49 sec.
68400 frames of binary, time 484.39 sec.
72000 frames of binary, time 512.72 sec.
75600 frames of binary, time 540.16 sec.
79200 frames of binary, time 568.12 sec.
82800 frames of binary, time 596.18 sec.
86400 frames of binary, time 623.65 sec.
90000 frames of binary, time 651.55 sec.
93600 frames of binary, time 679.71 sec.
97200 frames of binary, time 707.78 sec.
100800 frames of binary, time 726.51 sec.
104400 frames of binary, time 743.08 sec.
108000 frames of binary, time 760.31 sec.
111600 frames of binary, time 777.21 sec.
115200 frames of binary, time 794.06 sec.
118800 frames of binary, time 811.21 sec.
122400 frames of binary, time 827.99 sec.
126000 frames of binary, time 844.62 sec.
129600 frames of binary, time 861.72 sec.
133200 frames of binary, time 878.14 sec.
136800 frames of binary, time 895.02 sec.
140400 frames of binary, time 912.36 sec.
time 912.81 sec. Wrote 46800 frames per binary for 3 planes
>>>>>>>>>>>>>>>>>>>>> PLANE 0 <<<<<<<<<<<<<<<<<<<<<<
NOTE: not running registration, ops['do_registration']=0
binary path: 01_data/02_preprocessed_data/02_Psychedelics_L5b_suite2p/231113D_Sim-Cre/Saline_LSD/suite2p/plane0/data.bin
NOTE: applying default /home/meng/.suite2p/classifiers/classifier_user.npy
----------- ROI DETECTION
Binning movie in chunks of length 12
Binned movie of size [3900,512,512] created in 165.85 sec.
NOTE: FORCED spatial scale ~12 pixels, time epochs 3.25, threshold 48.75
0 ROIs, score=520.15
Detected 483 ROIs, 74.89 sec
After removing overlaps, 483 ROIs remain
----------- Total 241.48 sec.
----------- EXTRACTION
Masks created, 0.79 sec.
/home/meng/anaconda3/envs/2p/lib/python3.10/site-packages/suite2p/extraction/extract.py:125: NumbaTypeSafetyWarning: unsafe cast from uint64 to int64. Precision may be lost.
Fi[n] = np.dot(data[:, cell_ipix[n]], cell_lam[n])
Extracted fluorescence from 483 ROIs in 46800 frames, 191.22 sec.
----------- Total 192.25 sec.
----------- CLASSIFICATION
['npix_norm', 'compact', 'skew']
----------- SPIKE DECONVOLUTION
----------- Total 0.84 sec.
Plane 0 processed in 434.92 sec (can open in GUI).
>>>>>>>>>>>>>>>>>>>>> PLANE 1 <<<<<<<<<<<<<<<<<<<<<<
NOTE: not running registration, ops['do_registration']=0
binary path: 01_data/02_preprocessed_data/02_Psychedelics_L5b_suite2p/231113D_Sim-Cre/Saline_LSD/suite2p/plane1/data.bin
NOTE: applying default /home/meng/.suite2p/classifiers/classifier_user.npy
----------- ROI DETECTION
Binning movie in chunks of length 12
Binned movie of size [3900,512,512] created in 171.09 sec.
NOTE: FORCED spatial scale ~12 pixels, time epochs 3.25, threshold 48.75
0 ROIs, score=557.70
Detected 250 ROIs, 53.83 sec
After removing overlaps, 250 ROIs remain
----------- Total 225.38 sec.
----------- EXTRACTION
Masks created, 0.49 sec.
Extracted fluorescence from 250 ROIs in 46800 frames, 172.57 sec.
----------- Total 173.18 sec.
----------- CLASSIFICATION
['npix_norm', 'compact', 'skew']
----------- SPIKE DECONVOLUTION
----------- Total 0.42 sec.
Plane 1 processed in 399.10 sec (can open in GUI).
>>>> skipping flyback PLANE 2
total = 1747.42 sec.
Creating combined view
Traceback (most recent call last):
File "/media/meng/Data/Meng_Psychedelics/02_code/02_prism_soma_dendrite/run_suite2p.py", line 116, in <module>
output_ops = suite2p.run_s2p(ops=ops, db=db)
File "/home/meng/anaconda3/envs/2p/lib/python3.10/site-packages/suite2p/run_s2p.py", line 556, in run_s2p
io.combined(save_folder, save=True)
File "/home/meng/anaconda3/envs/2p/lib/python3.10/site-packages/suite2p/io/save.py", line 131, in combined
meanImgE[np.ix_(yrange, xrange)] = ops["meanImgE"]
KeyError: 'meanImgE'
Describe the issue:
Related to #1081. Rerunning suite2p on multiplane data where previously
ignore_flyback=2was set gives rise to an error that I believe occurs only because it is not taken into account that plane2 has to be ignored again.Reproduce the code example:
The easiest way to reproduce is to run suite2p on multiplane data and set
ignore_flyback>1andcombined=1. And simply run it another time. I run suite2p from the terminal.Error message:
Version information:
suite2p v0.14.3
Context for the issue:
No response