search

bahanonu / ciatah

CIAtah (pronounced cheetah): a software package for calcium imaging analysis of one- and two-photon imaging datasets. Documentation: https://git.io/ciatah_docs. Formerly known as calciumImagingAnalysis (ciapkg).
https://git.io/ciatah_docs
MIT License
80 stars 20 forks source link

signalSorter gone #98

Closed Antorithms closed 2 years ago

Antorithms commented 2 years ago

Hi Guys, First of all, thanks a lot for your hard work. I am working with peculiar videos and running ciatah for motioncorr and EXTRACTing calcium data separately to fine tune parameters for cell finding and refinement. I then would like to use signalSorter but it looks like it's not there in the repo. I hope I am not doing something stupid :') and thank you in advance for your help!!

Correction: it is in the ciapkg folder of the cloned repo so i guess it's just a problem on my side... Thanks anyways and sorry for bothering

Antonio

bahanonu commented 2 years ago

@Antorithms Good to hear from you. For using the signalSorter, can you provide the code you are using or steps (e.g. in the GUI) you are taking to try and run it? Also, which MATLAB version and OS are you running it on? That'll help me figure out how to get things working.

cheers!

Antorithms commented 2 years ago

Thanks a lot for the amazingly prompt reply!! Version is MATLAB R2021b.. I found signalSorter in the +ciapkg folder in the +api subfolder. all are in current path. When i try to run the function or simply open it for some awkward reason it does not find it.. I think it's just some glitch!

bahanonu commented 2 years ago

Sure thing! There are two ways to run it if you have the CIAtah root folder in the path.

If you are using the example from the EXTRACT repo (e.g. https://github.com/schnitzer-lab/EXTRACT-public#external-cell-checker), instead of signalSorter call the function with ciapkg.api.signalSorter or ciapkg.classification.signalSorter (CIAtah's function calls changed recently).

Else you can run with signalSorter if you import the CIAtah api package via the below code at the start of a script or function.

import ciapkg.api.* % import CIAtah functions in ciapkg package API.

Can also take a look at the examples at:

Antorithms commented 2 years ago

ciapkg.api.signalSorter worked! I gave it these defaults as iopts:

_iopts.inputMovie = '0000_00_00_p000_m1_NULL000_medFlt_detrend_treg_crop1.h5'; iopts.readMovieChunks = 1; % Binary: 1 = read movie from HDD, 0 = load entire movie into RAM iopts.inputDatasetName = '//1'; iopts.valid = 'neutralStart'; % all choices start out gray or neutral to not bias user iopts.cropSizeLength = 20; % region, in px, around a signal source for transient cut movies (subplot 2) iopts.cropSize = 20; % see above iopts.medianFilterTrace = 0; % whether to subtract a rolling median from trace iopts.subtractMean = 0; % whether to subtract the trace mean iopts.backgroundGood = [208,229,180]/255; iopts.backgroundBad = [244,166,166]/255; iopts.backgroundNeutral = repmat(230,[1 3])/255;

Nonetheless soon after starting it goes in a loop and shows many many time the error "Unrecognized function or variable 'instructionStr'" without even showing the gui.

Let me know and sorry about this! Antonio

bahanonu commented 2 years ago

Could you copy and paste the command window? That'll help me check what's going on (suspect it is a suptitle issue if bioinformatics toolbox is not installed).

There might be a toolbox missing, could you send the output of ciapkg.io.matlabToolboxCheck(); as that'll let me know if necessary Matlab toolboxes are installed (e.g. see https://github.com/bahanonu/ciatah#check-required-toolboxes-are-installed).

Antorithms commented 2 years ago

All toolboxes are installed!

Required toolboxes (if warning appears, fix or install requested toolbox). Toolbox license available and toolbox is installed! Parallel Computing Toolbox (Distrib_Computing_Toolbox). Toolbox license available and toolbox is installed! Image Processing Toolbox (Image_Toolbox). Toolbox license available and toolbox is installed! Signal Processing Toolbox (Signal_Toolbox). Toolbox license available and toolbox is installed! Statistics and Machine Learning Toolbox (Statistics_Toolbox).

2nd tier toolbox check (not required for main pre-processing pipeline). Toolbox license available and toolbox is installed! Computer Vision Toolbox (Video_and_Image_Blockset). Toolbox license available and toolbox is installed! Bioinformatics Toolbox (Bioinformatics_Toolbox). Toolbox license available and toolbox is installed! Financial Toolbox (Financial_Toolbox). Toolbox license available and toolbox is installed! Deep Learning Toolbox (Neural_Network_Toolbox).

Nonetheless, I get the suptitle error which does not stop it from running as well as the one from the screenshot attached hich basically goes over and over

image

Thanks a lot!!

bahanonu commented 2 years ago

Thanks, good to hear not a toolbox install issue. Could you send me a copy of the entire command window output (e.g. press Ctrl+C to kill the loop and copy the entire output)? That'll help me determine why the legend function isn't outputting instructionStr.

You mentioned still getting a suptitle error. If you run figure;subplot(1,2,1);plot(1);subplot(1,2,2);plot(1);suptitle('Hello world'); does that run without error?

Antorithms commented 2 years ago

Hey so, basically the loop makes Matlab crash and I cannot do anything but force close it. When i run obj = ciatah() i get the following

CIAtah (calciumImagingAnalysis)
A software package for analyzing one- and two-photon calcium imaging datasets.

Biafra Ahanonu <bahanonu@alum.mit.edu>
Version v4.0.4

Made in USA
✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ―――――――――――――――――――――――――
✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ―――――――――――――――――――――――――
✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ―――――――――――――――――――――――――
✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ―――――――――――――――――――――――――
✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ✶ ―――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――
―――――――――――――――――――――――――――――――――――――――――――――

******************************************
Constructing CIAtah imaging analysis object...
Removing unneeded directory from "to add" path list: extractor.m.
Removing unneeded directory from "to add" path list: normcorre.m.
Folders still need to be removed.
Removing unneeded directory from path: extractor.m.
Removing unneeded directory from path: normcorre.m.
Loading MIJI + ImageJ.
Adding to Java path: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\imagej\mij.jar
Loading MIJI + ImageJ.
Adding to Java path: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\imagej\ij.jar
Skipping loading of Miji.
******************************************
Removing all CNMF dir from path.
******************************************
Required toolboxes (if warning appears, fix or install requested toolbox).
Toolbox license available and toolbox is installed! Parallel Computing Toolbox (Distrib_Computing_Toolbox).
Toolbox license available and toolbox is installed! Image Processing Toolbox (Image_Toolbox).
Toolbox license available and toolbox is installed! Signal Processing Toolbox (Signal_Toolbox).
Toolbox license available and toolbox is installed! Statistics and Machine Learning Toolbox (Statistics_Toolbox).

2nd tier toolbox check (not required for main pre-processing pipeline).
Toolbox license available and toolbox is installed! Computer Vision Toolbox (Video_and_Image_Blockset).
Toolbox license available and toolbox is installed! Bioinformatics Toolbox (Bioinformatics_Toolbox).
Toolbox license available and toolbox is installed! Financial Toolbox (Financial_Toolbox).
Toolbox license available and toolbox is installed! Deep Learning Toolbox (Neural_Network_Toolbox).
******************************************
No folder paths input, run modelAddNewFolders method to add new folders.
No folder data specified, load data with modelVarsFromFiles method.
******************************************
Java max heap memory is 7.834 Gb, this should be sufficient to run Miji without problems. Please change "java.opts" file to increase heap space if run into Miji memory errors.
URL #1
Comparing
Local:  v4.0.4.
Online: v4.0.4.
Warning: Running most up-to-date version!
 Local version: v4.0.4 (2021.12.08 [22:12:10]).
 Online version v4.0.4 (2021.12.08 [22:12:10]).

> In ciapkg.io.updatePkg (line 83)
In ciatah (line 687) 
******************************************
Done initializing CIAtah!
******************************************

Run processing pipeline by typing below (or clicking link) into command window (no semi-colon!):
obj

When i run obj.setup() and select all dependencies I get this:


>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download Bio-Formats
=======
bfmatlab
Package already exists (and no forced update), skipping: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\bfmatlab.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download gramm (GRAMmar of graphics for Matlab, e.g. ggplot2-like)
=======
gramm
Package already exists (and no forced update), skipping: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\gramm.
Finished loading dependencies, now choose folders to add...
-------
Existing folders:
-------
Number new folders:1 | New folder indices: 1
Folders added:
C:\Data\falaanto\EXTRACT_tuning\M1_17
Adding file info to class...
>> figure;subplot(1,2,1);plot(1);subplot(1,2,2);plot(1);suptitle('Hello world');
Unrecognized function or variable 'suptitle'.

>> obj.setup();

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download Turboreg (motion correction)
=======
turboreg
Package already exists (and no forced update), skipping: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\turboreg.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download ImageJ
Directory already exists: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\imagej.
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\imagej\mij.jar
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\imagej\ij.jar
ImageJ already loaded!

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download CNMF, CNMF-E, and CVX code.
=======
CNMF-E
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\_downloads\cnmfe.zip
Unzipping file C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\_downloads\cnmfe.zip
Check internet connection or download and unzip files manually, see:
https://github.com/bahanonu/CNMF_E/archive/master.zip
https://github.com/flatironinstitute/CaImAn-MATLAB/archive/master.zip
http://web.cvxr.com/cvx/cvx-rd.zip

@@@@@@@
Error using ciapkg.download.downloadGithubRepositories (line 116)
An internal error has occurred.

Error in ciapkg.api.downloadGithubRepositories (line 6)
    [success] = ciapkg.download.downloadGithubRepositories('passArgs', varargin);

Error in ciapkg.download.downloadCnmfGithubRepositories (line 45)
        [success] = downloadGithubRepositories('options',optionsH);

Error in ciapkg.api.downloadCnmfGithubRepositories (line 6)
    [success] = ciapkg.download.downloadCnmfGithubRepositories('passArgs', varargin);

Error in ciapkg.io.loadDependencies (line 78)
                [success] = downloadCnmfGithubRepositories('forceUpdate',forceUpdate);

Error in ciatah/loadDependencies (line 60)
    ciapkg.io.loadDependencies('options',sopts);

Error in ciatah/setup (line 9)
    obj.loadDependencies;
@@@@@@@
Dialog box: Select cvx_setup.m (likely `C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\cvx_rd`
Check internet connection or download and unzip files manually, see:
https://github.com/bahanonu/CNMF_E/archive/master.zip
https://github.com/flatironinstitute/CaImAn-MATLAB/archive/master.zip
http://web.cvxr.com/cvx/cvx-rd.zip

@@@@@@@
Error using run (line 66)
 \  not found.

Error in ciapkg.download.downloadCnmfGithubRepositories (line 58)
            run(mfileToRun);

Error in ciapkg.api.downloadCnmfGithubRepositories (line 6)
    [success] = ciapkg.download.downloadCnmfGithubRepositories('passArgs', varargin);

Error in ciapkg.io.loadDependencies (line 78)
                [success] = downloadCnmfGithubRepositories('forceUpdate',forceUpdate);

Error in ciatah/loadDependencies (line 60)
    ciapkg.io.loadDependencies('options',sopts);

Error in ciatah/setup (line 9)
    obj.loadDependencies;
@@@@@@@

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download test one- and two photon datasets.
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\data\2014_04_01_p203_m19_check01\concat_recording_20140401_180333.h5
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\data\batch\2014_08_05_p104_m19_PAV08\concat_recording_20140805_162046.h5
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\data\batch\2014_08_06_p104_m19_PAV09\concat_recording_20140806_103546.h5
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\data\batch\2014_08_07_p104_m19_PAV10\concat_recording_20140807_102507.h5
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\data\twoPhoton\2017_04_16_p485_m487_runningWheel02\concat_recording_20140807_102507.h5

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download NWB (NeuroDataWithoutBorders)
=======
nwbpkg
Package already exists (and no forced update), skipping: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\nwbpkg.
NWB Schema types already loaded!

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download EXTRACT
=======
extract
Package already exists (and no forced update), skipping: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\extract.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download Bio-Formats
=======
bfmatlab
Package already exists (and no forced update), skipping: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\bfmatlab.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download gramm (GRAMmar of graphics for Matlab, e.g. ggplot2-like)
=======
gramm
Package already exists (and no forced update), skipping: C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\gramm.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download NoRMCorre (motion correction)
=======
normcorre
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\_downloads\normcorre.zip
Unzipping file C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\_downloads\normcorre.zip
Renaming C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\NoRMCorre-public-master to C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\normcorre 
Check internet connection or download and unzip files manually, see:
https://github.com/flatironinstitute/NoRMCorre/archive/master.zip

@@@@@@@
Error using movefile
No matching files named 'C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\NoRMCorre-public-master' were found.

Error in ciapkg.download.downloadGithubRepositories (line 124)
                movefile(oldDir,newDir);

Error in ciapkg.api.downloadGithubRepositories (line 6)
    [success] = ciapkg.download.downloadGithubRepositories('passArgs', varargin);

Error in ciapkg.io.loadDependencies (line 125)
                [success] = downloadGithubRepositories('options',optionsH);

Error in ciatah/loadDependencies (line 60)
    ciapkg.io.loadDependencies('options',sopts);

Error in ciatah/setup (line 9)
    obj.loadDependencies;
@@@@@@@

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Download Fiji (to run Miji)
Downloading Fiji (for use of Miji within Matlab)
=======
Fiji (for Miji)
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\_downloads\fiji-win64-20151222.zip
Already downloaded C:\Data\falaanto\EXTRACT_tuning\ciatah-master\_external_programs\fiji-win64-20151222
Miji loaded!

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Load Fiji/Miji into MATLAB path.
Miji loaded!
Finished loading dependencies, now choose folders to add...
Select a folder to add. Press cancel to stop adding folders.
Adding folder #1: C:\Data\falaanto\EXTRACT_tuning\M1_17.
Select a folder to add. Press cancel to stop adding folders.

Concerning suptitle here is what i get:

>> figure;subplot(1,2,1);plot(1);subplot(1,2,2);plot(1);suptitle('Hello world');
Unrecognized function or variable 'suptitle'.

Thanks a lot for your help.

bahanonu commented 2 years ago

Thanks, I'll push an update that'll deal with the looping issue and I'll call ciapkg.overloaded.suptitle, as that is a CIAtah version of suptitle, or sgtitle as those that should run even if by default suptitle can't be called by MATLAB (looks like Bioinformatics toolbox may have dropped suptitle in 2021a onward).

bahanonu commented 2 years ago

Fixed in https://github.com/bahanonu/ciatah/pull/100. Closing.