Suite2p -> CellReg footprint converter

[catubc]

Hello We are interested in aligning data in between many sessions and use Suite2P for cell registration.

1. Does your lab already have a function that extracts the Suite2p footprints to the required format?
2. Also, if we have 5 or more sessions, does the code to multi session alignment also?

Thanks so much, this looks like a great tool!

[Sheintuch]

Hi,

We do not have a function that converts the Suite2p footprints to the required format. I am not sure how it is organized in Suite2p, but what CellReg requires is a separate '.mat' file for each imaging session, with the corresponding spatial footprints (ROIs) of all detected cells. The matrix of the spatial footprints is of size NxMxK, where N is the number of neurons, M is the number of pixels in the y axis and K is the number of pixels in the x axis. Each entry in the matrix is equal to the corresponding pixel's value which represents its contribution to the overall cell's fluorescence (as obtained from different cell detection algorithms such as CNMF-e or PCA/ICA). If you need assistance in converting your data to the correct format feel free to send me the data from an example session, and I will send you the required commands.

Regarding the second question, you can register multiple sessions with CellReg. We have tested CellReg in the lab for as many as 20 sessions in the same mouse and it worked without any problems.

[catubc]

Hi @Sheintuch , thanks so much for the offer on 1. It would be really great if you could help with the conversion, I know there are many suite2p users who could benefit from easier loading.

I put the data here from 2 sessions:

https://drive.google.com/drive/folders/1or6-uTS6l68qMrxS1ETkKeSqNs_CseJn?usp=sharing

I am not great at Matlab, but I believe the individual footprints xy locations are in the following structures, where cell_id is a number:

x locations: stat.[cell_id].xpix y locations: stat.[cell_id].ypix

Thanks so much,

[Sheintuch]

I added a script _'formatconversion' that converts the output of either CNMF-e or Suite2p to the format of the input required by CellReg. In order to check that it works for your data you need to use 'git pull' to update your repository so that it is up to date with 'origin/master'. If you are not sure how to do that, simply download the repository again.

Please let me know if this script works for you so that I can tag the new CellReg version and close this issue. P.S. your data looks absolutely beautiful and could be registered with very high accuracy!

[catubc]

Hi @Sheintuch , thanks so much for this. I'll take a look at it shortly and update my comment. Indeed, the data is pretty good and with partially supervised methods we get about 50-60% overlap. I'm hoping your tool will give us somewhere close to that and skips the supervised step.

[catubc]

Hello. So we started to use the converter, there are 2 issues:

1. The data converter only works on lists of files (trying to convert a single file crashes). But this is not a big issue for us (we can leave it as is and warn other users in case they get confused).

2. The bigger issue is that the code crashes at the Register Cells step. My best guess is that the files we converted don't have a default "maximal_distance" parameter. Or something along these lines.

Any suggestions?

>> 
>> 
>> 
Reference to non-existent field 'maximal_distance'.

Error in CellReg>register_cells_initial_Callback (line 1010)
normalized_maximal_distance=data_struct.maximal_distance/microns_per_pixel;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_initial_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

[catubc]

Also, if we go back and try and load the aligned data using the Load aligned data function we also seem to be missing some parameters:

CellReg
Loading aligned data
Reference to non-existent field 'spatial_footprints_corrected'.

Error in CellReg>compute_model_Callback (line 805)
spatial_footprints_corrected=data_struct.spatial_footprints_corrected;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject)) 
Error using matlab.ui.internal.legacyMoveGUI (line 15)
Error while evaluating UIControl Callback.

Error using mkdir
Invalid path. The path must not contain a null character.

Error in CellReg>load_transformed_data_Callback (line 483)
        mkdir(figures_directory);

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('load_transformed_data_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

>> 

[catubc]

Ok, so I hardcoded the maximal_distance in the code, and then the next variable is missing also. So if I set that to some value manually (e.g. number_of_bins==10), then it does get past this step.

>> 
>> 
>> 
Reference to non-existent field 'number_of_bins'.

Error in CellReg>register_cells_initial_Callback (line 1012)
number_of_bins=data_struct.number_of_bins;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_initial_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

[Sheintuch]

Regarding the problem with converting only a single session, I will fix this. The rest of the issues seem to all be related to the same problem. It is not because of converted files since they are not supposed to include any information other than the spatial footprints.

Which CellReg version are you using?

[catubc]

Ok, i pulled the latest. I then loaded the previous aligned .mat file and still getting a couple of errors.

The first error is from saving during the prob model step. The second was after i clicked on "Register Cells"

I'll try again from scratch ...

Calculating a probabilistic model of the data

Local minimum possible.

lsqcurvefit stopped because the final change in the sum of squares relative to 
its initial value is less than the value of the function tolerance.

<stopping criteria details>
The spatial correlations model is best suited for the data
Warning: Variable 'hgS_070000' was not saved. For variables larger than 2GB use MAT-file version 7.3 or later. 
> In matlab.graphics.internal.figfile.FigFile/write (line 32)
  In savefig (line 83)
  In plot_models (line 123)
  In CellReg>compute_model_Callback (line 929)
  In gui_mainfcn (line 95)
  In CellReg (line 85)
  In matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject)) 
Error using save
Error closing file /media/cat/4TB/donato/DON-003343/tmep/updated/Figures/Stage 3 - model.fig.
The file may be corrupt.

Error in matlab.graphics.internal.figfile.FigFile/write (line 32)
save(obj.Path, obj.MatVersion, '-struct', 'SaveVars');

Error in savefig (line 83)
FF.write();

Error in plot_models (line 123)
savefig(fullfile(figures_directory,'Stage 3 - model.fig'))

Error in CellReg>compute_model_Callback (line 929)
plot_models(centroid_distances_model_parameters,NN_centroid_distances,NNN_centroid_distances,centroid_distances_distribution,centroid_distances_model_same_cells,centroid_distances_model_different_cells,centroid_distances_model_weighted_sum,centroid_distance_intersection,centers_of_bins,microns_per_pixel,normalized_maximal_distance,figures_directory,figures_visibility,spatial_correlations_model_parameters,NN_spatial_correlations,NNN_spatial_correlations,spatial_correlations_distribution,spatial_correlations_model_same_cells,spatial_correlations_model_different_cells,spatial_correlations_model_weighted_sum,spatial_correlation_intersection)

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

Reference to non-existent field 'maximal_distance'.

Error in CellReg>register_cells_initial_Callback (line 1025)
normalized_maximal_distance=data_struct.maximal_distance/microns_per_pixel;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_initial_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
Reference to non-existent field 'maximal_distance'.

Error in CellReg>register_cells_initial_Callback (line 1025)
normalized_maximal_distance=data_struct.maximal_distance/microns_per_pixel;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_initial_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

Exception in thread "AWT-EventQueue-0": java.lang.NullPointerException
    at com.mathworks.help.helpui.DocRoot.buildDocPageUrl(DocRoot.java:44)
    at com.mathworks.help.helpui.DocPageUrlBuilder.buildDocPageUrl(DocPageUrlBuilder.java:25)
    at com.mathworks.help.helpui.DocPageUrlBuilder.buildFileUrl(DocPageUrlBuilder.java:21)
    at com.mathworks.mlwidgets.help.LightweightHelpPanel$JsHelpPanelClientRunnable.getUrlToLoad(LightweightHelpPanel.java:212)
    at com.mathworks.mlwidgets.help.LightweightHelpPanel$JsHelpPanelClientRunnable.run(LightweightHelpPanel.java:196)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
>> 

[catubc]

Alright, so loading the converted files from scratch seems to crash even earlier:

Sorry about this, my matlab skills are pretty low, wish I could help debut...

>> 
Stage 1 - Loading sessions
Reference to non-existent field 'write2file_on'.

Error in CellReg>add_session_Callback (line 367)
    if get(handles.write2file_on,'Value');

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('add_session_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

>> 

[Sheintuch]

Yes. I am sorry about that. I updated CellReg to v1.4.7 today, but found out now there is a problem with new version. Please try and use v1.4.6, until I fix these problems, and let me know if it works.

[Sheintuch]

OK, I apologize for the confusion. First of all it turns out that I forgot, but there was already a function that converts the data from Suite2p to the CellReg format. The function is called 's2pToCellReg.m' and is in the helper folder. I also added now my new function to helper folder and I fixed the problem with reformatting a single file.

Regarding the other issues, I am pretty sure they were all because of usage of problematic versions. I now updated CellReg to a new version v1.4.8 that I also tested on your data and had no errors. Can you please check if the newest version works well for you and solves all the problems?

Thanks!

[catubc]

Ok, I'm sorry I started from a new install of v1.4.8, but now I can't even get past the initial alignment step:

setup
>> CellReg
Stage 1 - Loading sessions
Calculating spatial footprints projections:
Calculating footprints projections for session #1 -  done
Calculating footprints projections for session #2 -  done
Reference to non-existent field 'number_of_sessions'.

Error in CellReg>transform_sessions_Callback (line 620)
number_of_sessions=data_struct.number_of_sessions;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('transform_sessions_Callback',hObject,eventdata,guidata(hObject)) 
Error using savefig (line 36)
Error while evaluating UIControl Callback.

Done
>> 

[catubc]

And the original suite2p conversion function crashes at startup. Maybe the suite2p people changed the output format of their data?

This is not a rush to fix, obviously. I'm hoping the alignment could work for us first.

>> s2pToCellReg
Index exceeds the number of array elements (0).

Error in s2pToCellReg (line 17)
    maskOverlap = varargin{1};

>> s2pToCellReg('/media/cat/4TB/donato/DON-003343/tmep/allsessions/')
Error using load
Unable to read file '/media/cat/4TB/donato/DON-003343/tmep/allsessions/'. Input cannot be a directory.

Error in s2pToCellReg (line 22)
load(s2p_path,'stat','ops');

>> s2pToCellReg('/media/cat/4TB/donato/DON-003343/tmep/allsessions/Fall_1.mat')
Error using sub2ind (line 43)
Out of range subscript.

Error in s2pToCellReg (line 32)
        sub2ind([size(footprint,2) , size(footprint,3)] , ...

[Sheintuch]

Did you run the setup script after downloading cloning CellReg? This step adds the required directories to the MATLAB set path, which is critical to run the code. Another thing is that it might be required for you to change the MATLAB current folder to the CellReg directory. If this doesn't solve your problem, you can also try and run the cell registration procedure from the 'demo.m' script. Perhaps we can localize the problem in this way.

[catubc]

Ok, so demo runs smoothly:

>> demo
Stage 1 - Loading sessions
Calculating spatial footprints projections:
Calculating footprints projections for session #1 -  done
Calculating footprints projections for session #2 -  done
Calculating footprints projections for session #3 -  done
Calculating footprints projections for session #4 -  done
Calculating footprints projections for session #5 -  done
Done
Stage 2 - Aligning sessions
Normalizing spatial footprints:
Normalizing spatial footprints for session #1 -  done
Normalizing spatial footprints for session #2 -  done
Normalizing spatial footprints for session #3 -  done
Normalizing spatial footprints for session #4 -  done
Normalizing spatial footprints for session #5 -  done
Calculating spatial footprints projections:
Calculating footprints projections for session #1 -  done
Calculating footprints projections for session #2 -  done
Calculating footprints projections for session #3 -  done
Calculating footprints projections for session #4 -  done
Calculating footprints projections for session #5 -  done
Calculating centroid locations:
Calculating centroid locations for session #1 - 100%    [..........] done
Calculating centroid locations for session #2 - 100%    [..........] done
Calculating centroid locations for session #3 - 100%    [..........] done
Calculating centroid locations for session #4 - 100%    [..........] done
Calculating centroid locations for session #5 - 100%    [..........] done
Aligning session #2:
Checking for rotations
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 4).
No rotations required
Translating spatial footprints
Aligning session #3:
Checking for rotations
Rotating and translating spatial footprints
Aligning session #4:
Checking for rotations
No rotations required
Translating spatial footprints
Aligning session #5:
Checking for rotations
No rotations required
Translating spatial footprints
Parallel pool using the 'local' profile is shutting down.
Done
Stage 3 - Calculating a probabilistic model of the data
Calculating the distributions of cell-pair similarities:
Calculating spatial correlations and centroid distances for session #1 - 100%    [..........] done
Calculating spatial correlations and centroid distances for session #2 - 100%    [..........] done
Calculating spatial correlations and centroid distances for session #3 - 100%    [..........] done
Calculating spatial correlations and centroid distances for session #4 - 100%    [..........] done
Calculating spatial correlations and centroid distances for session #5 - 100%    [..........] done
Part a done
Calculating a probabilistic model of the data

Local minimum possible.

lsqcurvefit stopped because the final change in the sum of squares relative to 
its initial value is less than the value of the function tolerance.

<stopping criteria details>
The spatial correlations model is best suited for the data
Done
Stage 4 - Performing initial registration
Registering cells:
Initializing list with the cells from session #1
Registering cells in session #2 - 100%    [..........] done
Registering cells in session #3 - 100%    [..........] done
Registering cells in session #4 - 100%    [..........] done
Registering cells in session #5 - 100%    [..........] done
Calculating spatial footprints projections:
Calculating projections for session #1 - 100%    [..........] done
Calculating projections for session #2 - 100%    [..........] done
Calculating projections for session #3 - 100%    [..........] done
Calculating projections for session #4 - 100%    [..........] done
Calculating projections for session #5 - 100%    [..........] done
816 cells were found
Done
Stage 5 - Performing final registration
Clustering cells:
Performing clustering (iteration #1) -  done
Performing clustering (iteration #2) -  done
Calculating spatial footprints projections:
Calculating projections for session #1 - 100%    [..........] done
Calculating projections for session #2 - 100%    [..........] done
Calculating projections for session #3 - 100%    [..........] done
Calculating projections for session #4 - 100%    [..........] done
Calculating projections for session #5 - 100%    [..........] done
Saving the results
787 cells were found
End of cell registration procedure

[catubc]

Ok, so I re-tried running my 2 sessions right after the demo, and it now got through to the Alignment part.

My guess is that there were default variables in memory from the demo run to help the code get to the next stage.

So this error was no longer coming up. Even though I actually only have 2 sessions and the demo had 10 sessions. My guess is the code just needs some parameters to be initialized somewhere and we're missing that in the custom code.

Calculating footprints projections for session #2 -  done
Reference to non-existent field 'number_of_sessions'.

Error in CellReg>transform_sessions_Callback (line 620)
number_of_sessions=data_struct.number_of_sessions;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in

However I'm still getting errors for not being able to save the Figure:

<stopping criteria details>
The spatial correlations model is best suited for the data
Warning: Variable 'hgS_070000' was not saved. For variables larger than 2GB use MAT-file version 7.3 or later. 
> In matlab.graphics.internal.figfile.FigFile/write (line 32)
  In savefig (line 83)
  In plot_models (line 123)
  In CellReg>compute_model_Callback (line 914)
  In gui_mainfcn (line 95)
  In CellReg (line 85)
  In matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject)) 
Error using save
Error closing file /media/cat/4TB/donato/DON-003343/tmep/results/Figures/Stage 3 - model.fig.
The file may be corrupt.

Error in matlab.graphics.internal.figfile.FigFile/write (line 32)
save(obj.Path, obj.MatVersion, '-struct', 'SaveVars');

Error in savefig (line 83)
FF.write();

Error in plot_models (line 123)
savefig(fullfile(figures_directory,'Stage 3 - model.fig'))

Error in CellReg>compute_model_Callback (line 914)
plot_models(centroid_distances_model_parameters,NN_centroid_distances,NNN_centroid_distances,centroid_distances_distribution,centroid_distances_model_same_cells,centroid_distances_model_different_cells,centroid_distances_model_weighted_sum,centroid_distance_intersection,centers_of_bins,microns_per_pixel,normalized_maximal_distance,figures_directory,figures_visibility,spatial_correlations_model_parameters,NN_spatial_correlations,NNN_spatial_correlations,spatial_correlations_distribution,spatial_correlations_model_same_cells,spatial_correlations_model_different_cells,spatial_correlations_model_weighted_sum,spatial_correlation_intersection)

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

And also this field is still missing

>> 
>> 
>> 
>> 
Reference to non-existent field 'maximal_distance'.

Error in CellReg>register_cells_initial_Callback (line 1010)
normalized_maximal_distance=data_struct.maximal_distance/microns_per_pixel;

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_initial_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

[Sheintuch]

Are you running the same 2 sessions that you sent to me or is a different data set? Did you run the CellReg setup script?

[catubc]

It's the same 2 sessions. Yes, the CellReg_setup script was run.

I added the converted data to the folder above just in case we're not on the same page:

https://drive.google.com/drive/folders/1or6-uTS6l68qMrxS1ETkKeSqNs_CseJn?usp=sharing

[catubc]

Alright, so I hardcoded maximum_distance and number_of_bins and the GUI ran to Step 4. But for some reason, the file is not saving and I can't run step 5.

Registering cells:
Initializing list with the cells from session #1
Registering cells in session #2 - 100%    [..........] done
Warning: Variable 'hgS_070000' was not saved. For variables larger than 2GB use MAT-file version 7.3 or later. 
> In matlab.graphics.internal.figfile.FigFile/write (line 32)
  In savefig (line 83)
  In plot_initial_registration (line 63)
  In CellReg>register_cells_initial_Callback (line 1029)
  In gui_mainfcn (line 95)
  In CellReg (line 85)
  In matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_initial_Callback',hObject,eventdata,guidata(hObject)) 
Error using save
Error closing file /media/cat/4TB/donato/DON-003343/tmep/results/Figures/Stage 4 - same versus different cells.fig.
The file may be corrupt.

Error in matlab.graphics.internal.figfile.FigFile/write (line 32)
save(obj.Path, obj.MatVersion, '-struct', 'SaveVars');

Error in savefig (line 83)
FF.write();

Error in plot_initial_registration (line 63)
savefig(fullfile(figures_directory,'Stage 4 - same versus different cells.fig'))

Error in CellReg>register_cells_initial_Callback (line 1029)
        plot_initial_registration(cell_to_index_map,number_of_bins,spatial_footprints_corrected,initial_registration_type,figures_directory,figures_visibility,registered_cells_spatial_correlations,non_registered_cells_spatial_correlations)

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_initial_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
Error using CellReg>register_cells_final_Callback (line 1068)
Final registration cannot be performed before initial registration

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('register_cells_final_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

[Sheintuch]

It is problematic to hard code the parameters to solve all these issues. They are all related to the same thing so we need to get to the source of the problem and solve everything at once.

If the demo script works well for you, then first of all you can use this for now to register your data until we solve the problem with the GUI. No need for you to waste your time on fixing the code.

You can try and maybe download CellReg on a different computer and do the entire process again and see if it works. This is really weird as there are many people are using CellReg and I never heard of this specific problem before.

[catubc]

Here's the error screen grab

[Sheintuch]

This specific error is correct. You cannot run step 4 (initial registration) before running step 3 (compute probabilistic model). Unless you are referring here to something else that I am missing.

[catubc]

Yes, ok. But I did run that step, and it seemed to complete. Except maybe one of the rquired files is not saving?

> 
Stage 3 - Calculating a probabilistic model of the data
Calculating the distributions of cell-pair similarities:
Calculating spatial correlations and centroid distances for session #1 - 100%    [..........] done
Calculating spatial correlations and centroid distances for session #2 - 100%    [..........] done
Warning: A large portion of neighboring cells with negative spatial correlations (non-overlapping) were found. This could occur if the provided microns
per pixel ratio is incorrect or the maximal distance is too large. For 2-photon registration try reducing maximal distance to 10-12 microns 
> In compute_data_distribution (line 131)
  In CellReg>compute_model_Callback (line 849)
  In gui_mainfcn (line 95)
  In CellReg (line 85)
  In matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject)) 
Calculating a probabilistic model of the data

Local minimum possible.

lsqcurvefit stopped because the final change in the sum of squares relative to 
its initial value is less than the value of the function tolerance.

<stopping criteria details>
The spatial correlations model is best suited for the data
Warning: Variable 'hgS_070000' was not saved. For variables larger than 2GB use MAT-file version 7.3 or later. 
> In matlab.graphics.internal.figfile.FigFile/write (line 32)
  In savefig (line 83)
  In plot_models (line 123)
  In CellReg>compute_model_Callback (line 914)
  In gui_mainfcn (line 95)
  In CellReg (line 85)
  In matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject)) 
Error using save
Error closing file /media/cat/4TB/donato/DON-003343/tmep/results/Figures/Stage 3 - model.fig.
The file may be corrupt.

Error in matlab.graphics.internal.figfile.FigFile/write (line 32)
save(obj.Path, obj.MatVersion, '-struct', 'SaveVars');

Error in savefig (line 83)
FF.write();

Error in plot_models (line 123)
savefig(fullfile(figures_directory,'Stage 3 - model.fig'))

Error in CellReg>compute_model_Callback (line 914)
plot_models(centroid_distances_model_parameters,NN_centroid_distances,NNN_centroid_distances,centroid_distances_distribution,centroid_distances_model_same_cells,centroid_distances_model_different_cells,centroid_distances_model_weighted_sum,centroid_distance_intersection,centers_of_bins,microns_per_pixel,normalized_maximal_distance,figures_directory,figures_visibility,spatial_correlations_model_parameters,NN_spatial_correlations,NNN_spatial_correlations,spatial_correlations_distribution,spatial_correlations_model_same_cells,spatial_correlations_model_different_cells,spatial_correlations_model_weighted_sum,spatial_correlation_intersection)

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in CellReg (line 85)
    gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('compute_model_Callback',hObject,eventdata,guidata(hObject))

Error while evaluating UIControl Callback.

[catubc]

Ok, great, so I hardcoded 2 sessions into the demo seemed to have worked.

Not sure what the issues are with the GUI.

Do you have any advice re: demo parameters as I guess we're running with the defaults for now?

Thanks so much for all the help, and sorry for these weird crashes.

>> demo
Stage 1 - Loading sessions
Calculating spatial footprints projections:
Calculating footprints projections for session #1 -  done
Calculating footprints projections for session #2 -  done
Done
Stage 2 - Aligning sessions
Normalizing spatial footprints:
Normalizing spatial footprints for session #1 -  done
Normalizing spatial footprints for session #2 -  done
Calculating spatial footprints projections:
Calculating footprints projections for session #1 -  done
Calculating footprints projections for session #2 -  done
Calculating centroid locations:
Calculating centroid locations for session #1 - 100%    [..........] done
Calculating centroid locations for session #2 - 100%    [..........] done
Aligning session #2:
Checking for rotations
Rotating and translating spatial footprints
Parallel pool using the 'local' profile is shutting down.
Done
Stage 3 - Calculating a probabilistic model of the data
Calculating the distributions of cell-pair similarities:
Calculating spatial correlations and centroid distances for session #1 - 100%    [..........] done
Calculating spatial correlations and centroid distances for session #2 - 100%    [..........] done
Part a done
Calculating a probabilistic model of the data

Local minimum possible.

lsqcurvefit stopped because the final change in the sum of squares relative to 
its initial value is less than the value of the function tolerance.

<stopping criteria details>
The spatial correlations model is best suited for the data
Done
Stage 4 - Performing initial registration
Registering cells:
Initializing list with the cells from session #1
Registering cells in session #2 - 100%    [..........] done
Calculating spatial footprints projections:
Calculating projections for session #1 - 100%    [..........] done
Calculating projections for session #2 - 100%    [..........] done
1513 cells were found
Done
Stage 5 - Performing final registration
Clustering cells:
Performing clustering (iteration #1) -  done
Calculating spatial footprints projections:
Calculating projections for session #1 - 100%    [..........] done
Calculating projections for session #2 - 100%    [..........] done
Saving the results
1519 cells were found
End of cell registration procedure
>> 

[Sheintuch]

The same defaults are used in the GUI and the demo script. The only parameter you should probably change is the microns per pixel. How much is it in your recordings?

If you found 1519 cells than it is very similar to what I obtained for your data. I found 1515 cells while using 2.1 microns per pixel because I didn't know the true value in your data. This was with the rest of the parameters kept as their default values.

I am wondering what MATLAB version you are using. Perhaps you could try and run CellReg in an older MATLAB version. If that solves the problems with the GUI then it might be that there are changes in the new MATLAB version that I need to address so that CellReg will be suitable for the newer MATLAB versions.

[catubc]

Hello @Sheintuch , thanks for your patience. I finally finished a comparison analysis on 3 sequential imaging sessions (I initially tried 10 sessions, and seems your package worked but matlab died around stage 4 because of insufficient RAM; we are working on getting more).

The matches on the left are using ROIMatchPub which is this older (~3yrs old) partially supervised (click on features across images) package. Even when I relaxed the footprint overlap step to consider "convex hull" of each cell footprint (rather than the regular contour), CellReg is still much better.

Re: Matlab version, we have R2019a and newer versions, so perhaps that's part of the issues with using the GUI. If you have the time/energy to try and figure out the bugs, please ping me and walk me through the steps. But for the short/medium term I think I'll teach people at the lab to just use the demo script and set the parameters inline.

Thank you so much for your help, I will try to spread the news to our lab and others about the package.

[Sheintuch]

I am glad that it worked better with the demo script. You have very large numbers of cells so it is not that surprising that you had memory issues. Hopefully getting more RAM will solve this.

Regarding the MATLAB version and all the errors - it is not a problem with newer versions. To test the bugs you are finding I used my own laptop where nothing was installed in advance. Then, I performed the following steps: 1) I installed MATLAB 2021b with all the toolboxes/packages. 2) I cloned the CellReg repository. 3) I ran the _'CellRegsetup' script. 4) I loaded the converted spatial footprints (the ones in the dropbox folder you sent me) into CellReg. 5) I ran the entire CellReg procedure for your data.

Overall, I encountered no errors with registering your data. Since this problem is not reproduced I guess that this issue can be considered closed. Nonetheless, the errors you encountered are a real mystery for me, and I am very curious if others in your lab encounter the same errors or if they can use the GUI smoothly.

[catubc]

Thanks so much @Sheintuch , I will also try a fresh install on some of my colleagues computers later this week/early next. For now I'm more than happy with running the demo script with the main variables exposed and move to the top of the script. I'll let you know if we have issues again.

Thanks again for taking so much time to help us with this issue. I had another question re: running CellReg on Inscopix 1P detected footprints. But maybe I'll search a bit in the repo and if I don't find an answer start a separate issue to get advice.

Best

[Sheintuch]

You are very welcome. If you have any more questions, you are more than invited to join our slack channel: https://join.slack.com/t/cellreg/shared_invite/zt-12rsnsjfm-d2L4LO5HHtxZzJ0v_Dh7_g There you will find questions and comments about how to use CellReg. If for some reason you are unable to join our channel from this link, send me your email and I'll sent you a personal invitation.

[sarahapark]

Hello! I downloaded CellReg and ran CellReg_setup and cloned the repository. I am trying to register suite2p data and loaded the numbered suite2p files into CellReg using "Load New Data," but I am getting error messages. (see below) Where can I check that I have the "_formatconversion" script that you wrote to reorganize the suite2p data? Thank you!

---

Stage 1 - Loading sessions Conversion to double from struct is not possible.

Error in load_footprint_data (line 20) footprint(cell_n,:,:) = spatial_footprints_cell{cell_ID};

Error in load_multiple_sessions (line 19) temp_data = load_footprint_data(this_file_name);

Error in CellReg>load_new_data_Callback (line 271) [spatial_footprints,number_of_sessions]=load_multiple_sessions(file_names);

Error in gui_mainfcn (line 95) feval(varargin{:});

Error in CellReg (line 85) gui_mainfcn(gui_State, varargin{:});

Error in matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)CellReg('load_new_data_Callback',hObject,eventdata,guidata(hObject)) Error while evaluating UIControl Callback.

[sarahapark]

I found _format_conversion_Suite2p_CNMFe script in C:\Users\parks6\Documents\GitHub\CellReg\Helper. How should I load the suite2p data (Fall.mat) so that the script runs properly? Thank you!

Sarah

[Sheintuch]

Hi Sarah. Since I have never worked with Suite2p I am not sure about the specifics of how to load your data. Another issue is that I just recently started my postdoc and I don't have access to my previous work on fixing CellReg issues. One possibility is that other CellReg users (e.g., @catubc that opened this issue a year ago) could tell you how they solved the issue. You can also try the CellReg slack channel and check if anyone has already solved your issue in the past and can help you out.

In any case, I will work on getting access to my work on fixing CellReg issues from my previous lab (and on getting a new computer with MATLAB so I can test CellReg issues). But this might take a while so I hope you can solve the issue before that.

[sarahapark]

Hi @Sheintuch, I will try what you suggested. I need to create a Slack account to join CellReg using my work email but the @chop.edu domain isn't an option. Could I get an invitation from your workspace admin to parks6@chop.edu?

Thank you! Sarah

[sarahapark]

Hi @Sheintuch, I was able to convert the suite2p data set into CellReg format, and then successfully run CellReg on the newly formatted data! I basically ran the script "format_conversion_Suite2p_CNMFe" in MatLab's command window and then selected a Fall.mat file to convert. The output file is saved in the same directory as the original Fall.mat file and has a filename format of "converted(original filename)." In CellReg, I pressed "Load new data," selected multiple converted files, and went through each step of the registration process.

[Sheintuch]

Hi Sarah, It is great to hear that the data conversion script worked for you. I sent you an invitation to the CellReg slack channel.

[trose-neuro]

Hi @Sheintuch, great work with CellReg. We have the same goals for aligning cross-session suite2p data. Could you send me an invite as well? trose@uni-bonn.de

thx!