Closed willyh101 closed 9 months ago
via Masashi:
As you recommended, I used miniforge to re-install live2p and related packages with your yaml file in our scanimage PC. Then, we got a new error message, as you can see in the attached.
Caiman tried to reshape 84872000 data into 210944 x 500, then it caused an error, due to size mismatch, during the seed initialization step. Our x-y “Cut off” parameters in rig file and make mask 3d file is x:412, y:412 pixel, and we feed 500 frames for the initialization of caiman. So, it seems that caiman tried to reshape the (412 x 412) x 500 data points (84872000 data points) into (412 x 512) x 500 data points (210944 x 500).
Do you know if we need to specify “cut-off” parameters within caiman files somewhere, in addition to rig file and make mask 3d file? I assume that “cut off“ parameters are mismatched between rig file and caiman.
(live2p) C:\Users\scanimage>live2p -d 2
Welcome to live2p (v0.3)!
* Realtime, multiplane motion correction and source extraction using CaImAn OnACID, ScanImage, and websockets.
* A popup GUI will ask you to select you seed image by default.
* The seed image should be ~500 frames and will take ~ 30 seconds to process before you start the experiment.
Loading...
[INFO] Starting server...
[INFO] HOST=localhost
[INFO] PORT=5003
[INFO] Live2p websocket server ready!
[INFO] Connected to client 127.0.0.1
[INFO] Recieved setup data from SI
[INFO] nchannels set to 2
[INFO] nplanes set to 3
[INFO] fr set to 7.518024691358023
[INFO] folder set to F:\Data\Masashi\B24-1\Live2P_1
[INFO] Starting RealTimeWorker 0
00021591 - DEBUG - [workers:__init__:44] - Setting up params...
[INFO] Starting RealTimeWorker 1[INFO] Starting RealTimeWorker 2
00021593 - DEBUG - [workers:__init__:44] - Setting up params...
00021593 - DEBUG - [workers:__init__:44] - Setting up params...
00021624 - DEBUG - [workers:_setup_folders:115] - Set temp_path to F:\Data\Masashi\B24-1\Live2P_1\live2p\tmp
00021624 - DEBUG - [workers:_setup_folders:118] - Set out_path to F:\Data\Masashi\B24-1\Live2P_1\live2p\out
00021624 - DEBUG - [workers:_setup_folders:122] - Set working dir to F:\Data\Masashi\B24-1\Live2P_1\live2p\tmp
00021624 - DEBUG - [workers:__init__:180] - Max frames set to 128000
00021624 - DEBUG - [workers:_setup_folders:115] - Set temp_path to F:\Data\Masashi\B24-1\Live2P_1\live2p\tmp
00021624 - DEBUG - [workers:_setup_folders:115] - Set temp_path to F:\Data\Masashi\B24-1\Live2P_1\live2p\tmp
00021625 - DEBUG - [workers:_setup_folders:118] - Set out_path to F:\Data\Masashi\B24-1\Live2P_1\live2p\out
00021625 - DEBUG - [workers:_setup_folders:122] - Set working dir to F:\Data\Masashi\B24-1\Live2P_1\live2p\tmp
00021625 - DEBUG - [workers:_setup_folders:118] - Set out_path to F:\Data\Masashi\B24-1\Live2P_1\live2p\out
00021625 - DEBUG - [workers:__init__:180] - Max frames set to 128000
00021625 - DEBUG - [workers:_setup_folders:122] - Set working dir to F:\Data\Masashi\B24-1\Live2P_1\live2p\tmp
00021625 - DEBUG - [workers:__init__:180] - Max frames set to 128000
Plane 2: Found 96 sources from MM3D...
00022060 - INFO - [workers:__init__:216] - Starting live2p worker.
Plane 0: Found 96 sources from MM3D...
00022060 - INFO - [workers:initialize_onacid:234] - Starting new OnACID initialization for live2p.
00022063 - INFO - [workers:__init__:216] - Starting live2p worker.
00022063 - DEBUG - [workers:make_init_mmap:239] - Making init memmap...
00022064 - INFO - [workers:initialize_onacid:234] - Starting new OnACID initialization for live2p.
00022064 - DEBUG - [workers:make_init_mmap:239] - Making init memmap...
Plane 1: Found 96 sources from MM3D...
00022072 - INFO - [workers:__init__:216] - Starting live2p worker.
00022072 - INFO - [workers:initialize_onacid:234] - Starting new OnACID initialization for live2p.
00022072 - DEBUG - [workers:make_init_mmap:239] - Making init memmap...
00024647 - DEBUG - [workers:make_init_mmap:257] - Init mmap saved to initplane2_d1_412_d2_412_d3_1_order_C_frames_500_.mmap.
00024669 - DEBUG - [workers:make_init_mmap:257] - Init mmap saved to initplane1_d1_412_d2_412_d3_1_order_C_frames_500_.mmap.
00024715 - DEBUG - [workers:make_init_mmap:257] - Init mmap saved to initplane0_d1_412_d2_412_d3_1_order_C_frames_500_.mmap.
00042728 - ERROR - [server:handler:295] - Error in connection handler
Traceback (most recent call last):
File "C:\Users\scanimage\miniforge3\envs\live2p\lib\site-packages\websockets\legacy\server.py", line 293, in handler
await self.ws_handler(self, path)
File "c:\users\scanimage\live2p\live2p\websockets\server.py", line 130, in handle_incoming_ws
await self.route(payload)
File "c:\users\scanimage\live2p\live2p\websockets\server.py", line 181, in route
await self.handle_setup(data)
File "c:\users\scanimage\live2p\live2p\websockets\server.py", line 237, in handle_setup
self.workers = await asyncio.gather(*tasks)
File "C:\Users\scanimage\miniforge3\envs\live2p\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "c:\users\scanimage\live2p\live2p\websockets\server.py", line 273, in start_worker
self.params, self.qs[plane], Ain_path=self.Ain_path, **self.kwargs)
File "c:\users\scanimage\live2p\live2p\workers.py", line 220, in __init__
self.initialize_onacid()
File "c:\users\scanimage\live2p\live2p\workers.py", line 236, in initialize_onacid
self.acid = self._initialize_new(init_mmap)
File "c:\users\scanimage\live2p\live2p\utils.py", line 219, in wrapper_timer
value = func(*args, **kwargs)
File "c:\users\scanimage\live2p\live2p\workers.py", line 282, in _initialize_new
acid.initialize_online(T=self.num_frames_max)
File "C:\Users\scanimage\miniforge3\envs\live2p\lib\site-packages\caiman\source_extraction\cnmf\online_cnmf.py", line 1032, in initialize_online
gSig=self.params.get('init', 'gSig'), return_object=False)
File "C:\Users\scanimage\miniforge3\envs\live2p\lib\site-packages\caiman\source_extraction\cnmf\online_cnmf.py", line 1532, in seeded_initialization
Yr = np.reshape(Y, (Ain.shape[0], Y.shape[-1]), order='F')
File "<__array_function__ internals>", line 6, in reshape
File "C:\Users\scanimage\miniforge3\envs\live2p\lib\site-packages\numpy\core\fromnumeric.py", line 299, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "C:\Users\scanimage\miniforge3\envs\live2p\lib\site-packages\numpy\core\fromnumeric.py", line 58, in _wrapfunc
return bound(*args, **kwds)
ValueError: cannot reshape array of size 84872000 into shape (210944,500)
Possible causes:
Is it possible Y-cropping is not properly implemented downstream in the code? This could result in dimension mismatches down the road. I'm not sure if this is the same as the original issue, but we should try to start here anyway.
Added a hotfix 027ac89 for addition cli debugging. Please re-pull/clone the repo and send me the output of live2p -d 2
Hi Will, Thank you very much for your help.
1) I updated the live2p packages using, "pip install -e ." 2) I put the same Y-Cropping value that I assume you use.
Now, live2p seems to work properly, although I am not sure if all live2p functions work perfectly. At the end of the session (Push "Abort" button to stop "Loop" in Scanimage), I got the data.mat, other python files and three hdf5 files (realtime_results_plane_0.hdf5, realtime_results_plane_1.hdf5, realtime_results_plane_2.hdf5). Are these the final output files?
The following is the copy of the miniforge prompt running live2p at debug mode 2:
(base) C:\Users\scanimage>conda activate live2p
(live2p) C:\Users\scanimage>live2p -d 2
00000510 - DEBUG - [cli:main:52] - Will load settings using rigfile: frankenscope.py
00000510 - DEBUG - [cli:main:58] - ip: localhost
00000510 - DEBUG - [cli:main:58] - port: 6000
00000510 - DEBUG - [cli:main:58] - output_folder: F:/Data/Masashi/caiman_out
00000511 - DEBUG - [cli:main:58] - Ain_path: F:/Data/Masashi/caiman_temp/template/makeMasks3D_img.mat
00000511 - DEBUG - [cli:main:58] - xslice: slice(50, 462, None)
00000511 - DEBUG - [cli:main:58] - yslice: slice(0, 512, None)
00000511 - DEBUG - [cli:main:58] - num_frames_max: 128000
Welcome to live2p (v0.3.1)!
* Realtime, multiplane motion correction and source extraction using CaImAn OnACID, ScanImage, and websockets.
* A popup GUI will ask you to select you seed image by default.
* The seed image should be ~500 frames and will take ~ 30 seconds to process before you start the experiment.
Loading...
[INFO] Starting server...
[INFO] HOST=localhost
[INFO] PORT=6000
[INFO] Live2p websocket server ready!
[INFO] Connected to client 127.0.0.1
[INFO] Recieved setup data from SI
[INFO] nchannels set to 2
[INFO] nplanes set to 3
[INFO] fr set to 7.518024691358023
[INFO] folder set to F:\Data\Masashi\B24-1\Live2P_3
[INFO] Starting RealTimeWorker 0
00090328 - DEBUG - [workers:__init__:44] - Setting up params...
00090328 - INFO - [params:set:972] - Changing key decay_time in group data from 0.4 to 1.0
00090329 - INFO - [params:set:972] - Changing key p in group preprocess from 2 to 1
00090329 - INFO - [params:set:972] - Changing key nb in group init from 1 to 3
[INFO] Starting RealTimeWorker 100090329 - INFO - [params:set:972] - Changing key gSig in group init from [5, 5] to (7, 7)
00090329 - INFO - [params:set:972] - Changing key nb in group spatial from 1 to 3
[INFO] Starting RealTimeWorker 200090329 - INFO - [params:set:972] - Changing key p in group temporal from 2 to 1
00090330 - INFO - [params:set:972] - Changing key nb in group temporal from 1 to 3
00090330 - DEBUG - [workers:__init__:44] - Setting up params...
00090330 - INFO - [params:set:972] - Changing key init_method in group online from bare to seeded
00090330 - DEBUG - [workers:__init__:44] - Setting up params...
00090330 - INFO - [params:set:972] - Changing key decay_time in group data from 0.4 to 1.0
00090330 - INFO - [params:set:972] - Changing key expected_comps in group online from 500 to 300
00090331 - INFO - [params:set:972] - Changing key decay_time in group data from 0.4 to 1.0
00090331 - INFO - [params:set:972] - Changing key p in group preprocess from 2 to 1
00090331 - INFO - [params:set:972] - Changing key update_num_comps in group online from True to False
00090331 - INFO - [params:set:972] - Changing key p in group preprocess from 2 to 1
00090331 - INFO - [params:set:972] - Changing key nb in group init from 1 to 3
00090331 - INFO - [params:set:972] - Changing key update_freq in group online from 200 to 100
00090331 - INFO - [params:set:972] - Changing key nb in group init from 1 to 3
00090331 - INFO - [params:set:972] - Changing key gSig in group init from [5, 5] to (7, 7)
00090331 - INFO - [params:set:972] - Changing key normalize in group online from False to True
00090331 - INFO - [params:set:972] - Changing key gSig in group init from [5, 5] to (7, 7)
00090331 - INFO - [params:set:972] - Changing key nb in group spatial from 1 to 3
00090331 - INFO - [params:set:972] - Changing key simultaneously in group online from False to True
00090331 - INFO - [params:set:972] - Changing key nb in group spatial from 1 to 3
00090332 - INFO - [params:set:972] - Changing key p in group temporal from 2 to 1
00090332 - INFO - [params:set:972] - Changing key niter_rig in group motion from 1 to 2
00090332 - INFO - [params:set:972] - Changing key p in group temporal from 2 to 1
00090332 - INFO - [params:set:972] - Changing key nb in group temporal from 1 to 3
00090332 - INFO - [params:set:972] - Changing key nb in group temporal from 1 to 3
00090333 - INFO - [params:set:972] - Changing key init_method in group online from bare to seeded
00090332 - INFO - [params:set:972] - Changing key init_method in group online from bare to seeded
00090333 - INFO - [params:set:972] - Changing key expected_comps in group online from 500 to 300
00090333 - INFO - [params:set:972] - Changing key expected_comps in group online from 500 to 300
00090333 - INFO - [params:set:972] - Changing key update_num_comps in group online from True to False
00090333 - INFO - [params:set:972] - Changing key update_num_comps in group online from True to False
00090333 - INFO - [params:set:972] - Changing key update_freq in group online from 200 to 100
00090333 - INFO - [params:set:972] - Changing key update_freq in group online from 200 to 100
00090333 - INFO - [params:set:972] - Changing key normalize in group online from False to True
00090333 - INFO - [params:set:972] - Changing key normalize in group online from False to True
00090333 - INFO - [params:set:972] - Changing key simultaneously in group online from False to True
00090333 - INFO - [params:set:972] - Changing key simultaneously in group online from False to True
00090333 - INFO - [params:set:972] - Changing key niter_rig in group motion from 1 to 2
00090333 - INFO - [params:set:972] - Changing key niter_rig in group motion from 1 to 2
00090365 - DEBUG - [workers:_setup_folders:115] - Set temp_path to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp
00090365 - DEBUG - [workers:_setup_folders:115] - Set temp_path to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp
00090365 - DEBUG - [workers:_setup_folders:115] - Set temp_path to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp
00090366 - DEBUG - [workers:_setup_folders:118] - Set out_path to F:\Data\Masashi\B24-1\Live2P_3\live2p\out
00090366 - DEBUG - [workers:_setup_folders:118] - Set out_path to F:\Data\Masashi\B24-1\Live2P_3\live2p\out
00090366 - DEBUG - [workers:_setup_folders:122] - Set working dir to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp
00090366 - DEBUG - [workers:_setup_folders:122] - Set working dir to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp
00090366 - DEBUG - [workers:_setup_folders:118] - Set out_path to F:\Data\Masashi\B24-1\Live2P_3\live2p\out
00090366 - DEBUG - [workers:__init__:180] - Max frames set to 128000
00090366 - DEBUG - [workers:__init__:180] - Max frames set to 128000
00090366 - DEBUG - [workers:_setup_folders:122] - Set working dir to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp
00090366 - DEBUG - [workers:__init__:180] - Max frames set to 128000
Plane 2: Found 73 sources from MM3D...
00090659 - INFO - [workers:__init__:216] - Starting live2p worker.
00090660 - INFO - [workers:initialize_onacid:234] - Starting new OnACID initialization for live2p.
00090660 - DEBUG - [workers:make_init_mmap:239] - Making init memmap...
Plane 0: Found 78 sources from MM3D...
00090684 - INFO - [workers:__init__:216] - Starting live2p worker.
00090684 - INFO - [workers:initialize_onacid:234] - Starting new OnACID initialization for live2p.
00090684 - DEBUG - [workers:make_init_mmap:239] - Making init memmap...
Plane 1: Found 85 sources from MM3D...
00090696 - INFO - [workers:__init__:216] - Starting live2p worker.
00090697 - INFO - [workers:initialize_onacid:234] - Starting new OnACID initialization for live2p.
00090697 - DEBUG - [workers:make_init_mmap:239] - Making init memmap...
00092081 - INFO - [params:set:972] - Changing key init_batch in group online from 200 to 500
00092377 - INFO - [params:set:972] - Changing key init_batch in group online from 200 to 500
00092568 - INFO - [params:set:972] - Changing key init_batch in group online from 200 to 500
00093805 - DEBUG - [workers:make_init_mmap:257] - Init mmap saved to initplane2_d1_512_d2_412_d3_1_order_C_frames_500_.mmap.
00093895 - INFO - [params:set:972] - Changing key fnames in group data from None to initplane2_d1_512_d2_412_d3_1_order_C_frames_500_.mmap
00093913 - DEBUG - [workers:make_init_mmap:257] - Init mmap saved to initplane1_d1_512_d2_412_d3_1_order_C_frames_500_.mmap.
00093935 - DEBUG - [workers:make_init_mmap:257] - Init mmap saved to initplane0_d1_512_d2_412_d3_1_order_C_frames_500_.mmap.
00093962 - INFO - [params:set:972] - Changing key fnames in group data from None to initplane1_d1_512_d2_412_d3_1_order_C_frames_500_.mmap
00093986 - INFO - [params:set:972] - Changing key fnames in group data from None to initplane0_d1_512_d2_412_d3_1_order_C_frames_500_.mmap
00097021 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00097179 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00097461 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00103235 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00103772 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00103820 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00108922 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00109952 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00110206 - WARNING - [movies:extract_shifts:371] - Movie average is negative. Removing 1st percentile.
00117371 - INFO - [online_cnmf:initialize_online:967] - Frame size:(512, 412)
00118924 - INFO - [online_cnmf:initialize_online:967] - Frame size:(512, 412)
00119274 - INFO - [online_cnmf:initialize_online:967] - Frame size:(512, 412)
00128550 - INFO - [params:set:972] - Changing key expected_comps in group online from 300 to 78
00129042 - INFO - [online_cnmf:_prepare_object:233] - Expecting 78 components
00129075 - INFO - [params:set:972] - Changing key expected_comps in group online from 300 to 73
00129575 - INFO - [online_cnmf:_prepare_object:233] - Expecting 73 components
00130624 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00130626 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00130627 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00130715 - INFO - [utils:recursively_save_dict_contents_to_group:511] - Ab is sparse ****
00131025 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00131027 - INFO - [utils:recursively_save_dict_contents_to_group:511] - A is sparse ****
00131031 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00131036 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00131036 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00131051 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00131156 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key ind_A is not saved.
00131157 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_in is not saved.
00131157 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_out is not saved.
00131157 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key loaded_model is not saved.
00131159 - INFO - [workers:save_acid:471] - Saved OnACID hdf5 to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp\realtime_init_plane_0.hdf5
00131159 - DEBUG - [workers:_initialize_new:288] - OnACID initialized.
00131159 - INFO - [utils:wrapper_timer:222] - <live2p.workers._initialize_new> done in 37.172s
00131259 - INFO - [params:set:972] - Changing key expected_comps in group online from 300 to 85
00131684 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00131686 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00131686 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00131764 - INFO - [utils:recursively_save_dict_contents_to_group:511] - Ab is sparse ****
00131875 - INFO - [online_cnmf:_prepare_object:233] - Expecting 85 components
00132077 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00132079 - INFO - [utils:recursively_save_dict_contents_to_group:511] - A is sparse ****
00132084 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00132086 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00132086 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00132098 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00132199 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key ind_A is not saved.
00132199 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_in is not saved.
00132200 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_out is not saved.
00132200 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key loaded_model is not saved.
00132201 - INFO - [workers:save_acid:471] - Saved OnACID hdf5 to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp\realtime_init_plane_2.hdf5
00132202 - DEBUG - [workers:_initialize_new:288] - OnACID initialized.
00132202 - INFO - [utils:wrapper_timer:222] - <live2p.workers._initialize_new> done in 38.306s
00133184 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00133186 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00133186 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00133270 - INFO - [utils:recursively_save_dict_contents_to_group:511] - Ab is sparse ****
00133559 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00133560 - INFO - [utils:recursively_save_dict_contents_to_group:511] - A is sparse ****
00133564 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00133565 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00133566 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00133578 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00133661 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key ind_A is not saved.
00133662 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_in is not saved.
00133662 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_out is not saved.
00133662 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key loaded_model is not saved.
00133664 - INFO - [workers:save_acid:471] - Saved OnACID hdf5 to F:\Data\Masashi\B24-1\Live2P_3\live2p\tmp\realtime_init_plane_1.hdf5
00133664 - DEBUG - [workers:_initialize_new:288] - OnACID initialized.
00133664 - INFO - [utils:wrapper_timer:222] - <live2p.workers._initialize_new> done in 39.702s
[INFO] Ready to process online!
00222139 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00222172 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00222174 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00224387 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00224407 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00224513 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00238421 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00238453 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00238474 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00240493 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00240573 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00240697 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00276783 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00276797 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00276803 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00276896 - INFO - [workers:process_frame_from_queue:360] - Total of 1000 frames processed. (Queue 0)
00276898 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 21 ms. (46.17 Hz) (Queue 0)
00276924 - INFO - [workers:process_frame_from_queue:360] - Total of 1000 frames processed. (Queue 1)
00276924 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (44.44 Hz) (Queue 1)
00276932 - INFO - [workers:process_frame_from_queue:360] - Total of 1000 frames processed. (Queue 2)
00276933 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 21 ms. (45.58 Hz) (Queue 2)
00279098 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00279104 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00279158 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00289797 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00289802 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00289807 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00306337 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00306339 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00306340 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00308495 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00308531 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00308586 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00319873 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00319912 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00319949 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00319979 - INFO - [workers:process_frame_from_queue:360] - Total of 1500 frames processed. (Queue 0)
00319980 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (45.42 Hz) (Queue 0)
00320026 - INFO - [workers:process_frame_from_queue:360] - Total of 1500 frames processed. (Queue 2)
00320027 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (44.93 Hz) (Queue 2)
00320062 - INFO - [workers:process_frame_from_queue:360] - Total of 1500 frames processed. (Queue 1)
00320063 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (43.96 Hz) (Queue 1)
00334320 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00334366 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00334448 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00355148 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00355192 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00355267 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00370125 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00370138 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00370142 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00403142 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00403147 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00403148 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00405342 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00405406 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00405455 - INFO - [workers:process_frame_from_queue:360] - Total of 2000 frames processed. (Queue 2)
00405456 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (44.48 Hz) (Queue 2)
00405500 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00405515 - INFO - [workers:process_frame_from_queue:360] - Total of 2000 frames processed. (Queue 0)
00405516 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (44.71 Hz) (Queue 0)
00405616 - INFO - [workers:process_frame_from_queue:360] - Total of 2000 frames processed. (Queue 1)
00405617 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 23 ms. (43.35 Hz) (Queue 1)
00407604 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00407623 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00407868 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00419778 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00419778 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00419865 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00447470 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00447473 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00447473 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00449546 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00449606 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00449669 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00491749 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00491768 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00491778 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00491871 - INFO - [workers:process_frame_from_queue:360] - Total of 2500 frames processed. (Queue 2)
00491871 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (45.06 Hz) (Queue 2)
00491875 - INFO - [workers:process_frame_from_queue:360] - Total of 2500 frames processed. (Queue 0)
00491875 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (45.18 Hz) (Queue 0)
00491885 - INFO - [workers:process_frame_from_queue:360] - Total of 2500 frames processed. (Queue 1)
00491886 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (43.87 Hz) (Queue 1)
00494034 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00494041 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00494185 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00496124 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00496178 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00496474 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00498385 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00498430 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00498777 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00521869 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00521917 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00521977 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00523950 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00523998 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00524056 - INFO - [workers:process_frame_from_queue:360] - Total of 3000 frames processed. (Queue 2)
00524056 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (45.22 Hz) (Queue 2)
00524110 - INFO - [workers:process_frame_from_queue:360] - Total of 3000 frames processed. (Queue 0)
00524111 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (45.31 Hz) (Queue 0)
00524185 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00524309 - INFO - [workers:process_frame_from_queue:360] - Total of 3000 frames processed. (Queue 1)
00524309 - INFO - [workers:process_frame_from_queue:364] - Average processing time: 22 ms. (43.87 Hz) (Queue 1)
00544906 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00544906 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00545000 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00547285 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00547327 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
00547491 - INFO - [online_cnmf:fit_next:769] - Updating Shapes
[INFO] Recieved acqAbort. Workers will continue running until all frames are completed.
00560014 - INFO - [workers:process_frame_from_queue:381] - Stopping live2p....
00560014 - INFO - [workers:process_frame_from_queue:381] - Stopping live2p....
00560014 - INFO - [workers:process_frame_from_queue:381] - Stopping live2p....
00560014 - DEBUG - [workers:process_frame_from_queue:384] - Processing done at: 20:59:20
00560014 - DEBUG - [workers:process_frame_from_queue:384] - Processing done at: 20:59:20
00560014 - DEBUG - [workers:process_frame_from_queue:384] - Processing done at: 20:59:20
00560014 - INFO - [workers:process_frame_from_queue:385] - Getting final results...
00560014 - INFO - [workers:process_frame_from_queue:385] - Getting final results...
00560014 - INFO - [workers:process_frame_from_queue:385] - Getting final results...
00560264 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00560268 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00560268 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00560351 - INFO - [utils:recursively_save_dict_contents_to_group:511] - Ab is sparse ****
00560808 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00560810 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00560810 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00560863 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00560863 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00560866 - INFO - [utils:recursively_save_dict_contents_to_group:511] - A is sparse ****
00560868 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00560909 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00560909 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00560911 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00560912 - INFO - [utils:recursively_save_dict_contents_to_group:511] - Ab is sparse ****
00560954 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00561072 - INFO - [utils:recursively_save_dict_contents_to_group:511] - Ab is sparse ****
00561138 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00561618 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key ind_A is not saved.
00561618 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00561619 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_in is not saved.
00561629 - INFO - [utils:recursively_save_dict_contents_to_group:511] - A is sparse ****
00561629 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_out is not saved.
00561634 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00561709 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00561709 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00561685 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key loaded_model is not saved.
00561725 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00561728 - INFO - [workers:save_acid:471] - Saved OnACID hdf5 to F:\Data\Masashi\B24-1\Live2P_3\live2p\out\realtime_results_plane_0.hdf5
00561810 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00561815 - INFO - [utils:recursively_save_dict_contents_to_group:511] - A is sparse ****
00561820 - INFO - [utils:recursively_save_dict_contents_to_group:471] - g is an object type
00561822 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key merged_ROIs is not saved.
00562503 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key coordinates is not saved.
00562525 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key groups is not saved.
00563171 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key ind_A is not saved.
00563172 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_in is not saved.
00563172 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_out is not saved.
00563172 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key loaded_model is not saved.
00563175 - INFO - [workers:save_acid:471] - Saved OnACID hdf5 to F:\Data\Masashi\B24-1\Live2P_3\live2p\out\realtime_results_plane_2.hdf5
00564530 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key ind_A is not saved.
00564531 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_in is not saved.
00564531 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key tf_out is not saved.
00564532 - INFO - [utils:recursively_save_dict_contents_to_group:481] - Key loaded_model is not saved.
00564534 - INFO - [workers:save_acid:471] - Saved OnACID hdf5 to F:\Data\Masashi\B24-1\Live2P_3\live2p\out\realtime_results_plane_1.hdf5
[INFO] Processing and saving final data.
[INFO] Live2p finished. Shutting down server.
(live2p) C:\Users\scanimage>
Glad it's working. Yes, those are the final output files.
Be 100% sure the dimensions in your makeMasks3D script match what you put into live2p, otherwise you will have very weird errors pop up. They can be hard to diagnose. I'm not sure what the exact cause of the original error was, but if it happens again (or you have a new problem) please create another issue here with as much information as you can provide and I'll help you sort it out.
For now, I'll close this issue as it's seemingly fixed itself...
Yes, those are the final output files.
OK, I got it. It is nice that live2p is working properly now.
Be 100% sure the dimensions in your makeMasks3D script match what you put into live2p
I will check the makeMask3D script and rig file again, just in case. Thanks again for your kind help!
Masashi
New issue, might be related to caiman update. Appears to only be an issue on recent installs of live2p. Live2p raises
OSError: Unable to synchronously create file (unable to truncate a file which is already open)
while trying to save a h5df file with the same name (each plane is getting the same file name and therefore fails) for OnACID processing, but Caiman should be able to read the mmap directly.We should try to disable saving in params or see what is causing the intermediate h5df file to be saved- since there is already a mmap file, it's unclear why a h5 is also being written to disk.
Verbose output from live2p: