'Sorting_analyzer' is not defined

[taningh86]

Hi! I was trying to use one of the submodules in the quality metrics module - compute_synchrony_metrics but ran into sorting_analyzer not defined error. Had a chat with @alejoe91 after that where he suggested to install 0.100.0 from the main to be able to use 'sorting_analyzer'. So I cloned from the source and ran the code below again, but the same error continues. Any suggestion how to get around this? import spikeinterface.qualitymetrics as sqm

synchrony = sqm.compute_synchrony_metrics(sorting_analyzer=sorting_analyzer, synchrony_sizes=(2, 4, 8))

[h-mayorquin]

Original question: Did you installed spikeinterfaced from the local cloned files?

in the enviroment where you are running your code, go to the folder where the code is and run pip install .

EDIT: @taningh86 solved the issue it by doing the following:

So I uninstalled it using pip uninstall spikeinterface in the conda terminal. Then I git cloned it from the source again and did pip install -e .. So that worked. For version 0.101.0 to work a new notebook has to be opened and any other notebooks that were working under the older 0.100.0 won't upgrade to 0.101.0 even after restarting the kernel.

[taningh86]

Hi @h-mayorquin . I cloned it from the source using conda terminal git clone https://github.com/SpikeInterface/spikeinterface.git cd spikeinterface pip install -e . cd ..

Are you suggesting that I should do pip install spikeinterface in terminal?

[h-mayorquin]

I don't know what you mean from the terminal.

That last command that you wrote within the spikeinterface folder seems correct:

``pip install .

I just wanted to verify that you installed the latest version.

You can confirm that your code is running the latest version by running:

import spikeinterface

spikeinterface.__version__

Maybe there is another problem

[zm711]

Also @taningh86 can you post

sorting_analyzer

So we can make sure you have an appropriate sorting_analyzer. The way I'm reading it; it seems like you haven't made your sorting_analyzer yet.

[taningh86]

I don't know what you mean from the terminal.

That last command that you wrote within the spikeinterface folder seems correct:

``pip install .

I just wanted to verify that you installed the latest version.

You can confirm that your code is running the latest version by running:

import spikeinterface

spikeinterface.__version__

Maybe there is another problem

I have version 0.100.0

[taningh86]

Also @taningh86 can you post

sorting_analyzer

So we can make sure you have an appropriate sorting_analyzer. The way I'm reading it; it seems like you haven't made your sorting_analyzer yet.

Hi @zm711 I believe you are right. I haven't made and not sure how to. I did look for instructions in the documentation but could not find it. I believe that's the reaosn why I am getting the error. Could you share an example of creating a sorting_analyzer?

[zm711]

We are getting ready to update the docs : #2563.

sorting_analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format='memory') # for testing stuff
sorting_analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format='binary_folder') # for saving stuff or format = 'zarr' for saving with compression

I would look at the PR I linked which includes the new docs for fine tuning other parameters (for example sparsity).

[h-mayorquin]

I don't know what you mean from the terminal. That last command that you wrote within the spikeinterface folder seems correct: ``pip install . I just wanted to verify that you installed the latest version. You can confirm that your code is running the latest version by running:

import spikeinterface

spikeinterface.__version__

Maybe there is another problem

I have version 0.100.0

Mmm I think that you should have version 0.101.0 when pip installing from source.

[taningh86]

We are getting ready to update the docs : #2563.

sorting_analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format='memory') # for testing stuff
sorting_analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format='binary_folder') # for saving stuff or format = 'zarr' for saving with compression

I would look at the PR I linked which includes the new docs for fine tuning other parameters (for example sparsity).

Thanks! This is helpful and I will explore the PR for other parameters.

[taningh86]

I don't know what you mean from the terminal. That last command that you wrote within the spikeinterface folder seems correct: ``pip install . I just wanted to verify that you installed the latest version. You can confirm that your code is running the latest version by running:

import spikeinterface

spikeinterface.__version__

Maybe there is another problem

I have version 0.100.0

Mmm I think that you should have version 0.101.0 when pip installing from source.

When I pip installed from the source it said following so it should be 0.101.0 but I am not sure why it's still shows version 0.100.0? after I installed I also restarted the kernel. Not sure why this is. Successfully installed neo-0.13.0 probeinterface-0.2.21 spikeinterface-0.101.0 threadpoolctl-3.3.0

[taningh86]

We are getting ready to update the docs : #2563.

sorting_analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format='memory') # for testing stuff
sorting_analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format='binary_folder') # for saving stuff or format = 'zarr' for saving with compression

I would look at the PR I linked which includes the new docs for fine tuning other parameters (for example sparsity).

I am getting "AttributeError: module 'spikeinterface' has no attribute 'create_sorting_analyzer". Any clue why this is? I wonder if this is due to the version issue I am having?

[zm711]

Yeah if you're installing version 0.100.0 that is the last version without the sorting analyzer. So in that case you would do the older version of waveform extractor and then do the compute synchrony with the waveform extractor instead of the sorting analyzer.

Do you want to try

pip uninstall spikeinterface

Then pull it down from git and do the pip install again and let us know the version number.

[taningh86]

Hi @zm711 Now I have version 0.101.0 I know the team is working on the documentation currently and its not there in the page yet. Looks like many of the sub-modules have been removed or their names are changed. Below is the usual sub-modules I import but getting an error that they don't exist import spikeinterface.extractors as se import spikeinterface.preprocessing as spre import spikeinterface.sorters as ss import spikeinterface.postprocessing as spost import spikeinterface.qualitymetrics as sqm import spikeinterface.comparison as sc import spikeinterface.exporters as sexp import spikeinterface.widgets as sw

I would appreciate if you can guide me to a quick recipe of the new sub-module terms. A short tutorial with 0.101.0 would be great if time allows for any member in the admin group.

Thanks Jimmy

[zm711]

Howdy Jimmy!

No modules have had their names changed. Instead the waveform_extractor is being switched over to the sorting_analyzer. Could you post the exact error you get. Let's start with: EDIT: had a typo in my import sorry!

import spikeinterface as si

Then try one module at a time. If you are getting an error on each module then it may be a broken install.

[taningh86]

all the sub modules were imported after import spikeinterface as si :) Let me try out one at a time and update you.

[taningh86]

ok. it worked now. I also tried importing a bunch at a time and it also worked. Could we keep this issue open for a bit until I get to the synchrony_metrics and provide updates/questions?

[zm711]

Of course. No worries. :)

[taningh86]

Hi, I am starting to get a hang of SortingAnalyzer and exploring all the available extensions. Here's what I did to create a sorting_analyzer and explore some of the extensions from spikeinterface import create_sorting_analyzer, load_sorting_analyzer sorting_analyzer = si.create_sorting_analyzer( sorting=sorting_wout_excess_spikes, recording=recording_saved, format='binary_folder', folder= 'sorting_analyzer' ) job_kwargs = dict(n_jobs=8, chunk_duration="1s", progress_bar=True) sorting_analyzer.compute( "random_spikes", method="uniform", max_spikes_per_unit=500, ) sorting_analyzer.compute("waveforms", ms_before=1.0, ms_after=2.0, return_scaled=True, **job_kwargs) sorting_analyzer.compute("templates", operators=["average", "median", "std"]) print(sorting_analyzer) sparsity = si.compute_sparsity(sorting_analyzer, method = "radius", radius_um = 30) w = sw.plot_unit_templates(sorting_analyzer,templates_percentile_shading=None,backend = "ipywidgets")

Now, the issue what I am seeing is 'very slow' processing of the waveforms- I mean going from one unit to another nce the plot is loaded takes more than 2 minutes. We had this discussion before and I was suggested to use templates_percentile-shading=None, which I did here. With the 0.100.0 version that took care of the slow processing, but with 0.101.0 it did not help. Can anything else be done about this?

[h-mayorquin]

ok. it worked now. I also tried importing a bunch at a time and it also worked. Could we keep this issue open for a bit until I get to the synchrony_metrics and provide updates/questions?

Hey, it is fine to keep the issue open. But from the point of view of the community it is better if you open a different issue for each question that you have. That way, when people use the google search or search within issues they will find more indicative titles and the information will be easier to fetch.

Could you do that for other questions you might have?

[h-mayorquin]

Now, the issue what I am seeing is 'very slow' processing of the waveforms- I mean going from one unit to another nce the plot is loaded takes more than 2 minutes. We had this discussion before and I was suggested to use templates_percentile-shading=None, which I did here. With the 0.100.0 version that took care of the slow processing, but with 0.101.0 it did not help. Can anything else be done about this?

This for example should be a different issue.

[h-mayorquin]

Also, can you write what solved your issue of not having the correct version of spikeinterface installed. That would be useful if someone gets the same problem that you did.

[taningh86]

Hi @h-mayorquin I agree with you. But I raised this issue already a week or so ago https://github.com/SpikeInterface/spikeinterface/issues/2554 and I was suggested to use templates_percentile_shading=None, whch worked for me then. But I think with version 0.101.0 it is not working. @zm711 and @alejoe91 were in conversation for that issue and, since, are already aware of it so I thought I will ask here.

[h-mayorquin]

Issues are better if they are more specific: "Processing of waveforms slower with shorter analyzerin version x" and then adding details that template_percentile_shading=None does not work and some more context seems good to me.

I am aware that this is harder for you, writing issues is work, that's why I am appealing to your sense of duty to the community. I also feel that our work answering your questions becomes more useful that way.

[taningh86]

For sure @h-mayorquin .I totally understand your point and I will create the issue right now! Thanks!

[taningh86]

Hi @h-mayorquin I agree with you. But I raised this issue already a week or so ago #2554 and I was suggested to use templates_percentile_shading=None, whch worked for me then. But I think with version 0.101.0 it is not working. @zm711 and @alejoe91 were in conversation for that issue and, since, are already aware of it so I thought I will ask here.

sorry I shared the wrong issue. This is the issue I raised earlier and was resolved https://github.com/SpikeInterface/spikeinterface/issues/2540

[zm711]

Also, can you write what solved your issue of not having the correct version of spikeinterface installed. That would be useful if someone gets the same problem that you did.

Also Jimmy like @h-mayorquin said sharing how you got the import to work would be super valuable here. Did you just restart your terminal? Restart your computer? Do something different with git. We have plenty of people on all sorts of OSs and conda vs pip trying to get this to work. So your experience would be super valuable so that if someone runs into this they can try your solution :)

[h-mayorquin]

I saw that you were using editable installs. Be warned that after setuptools was dropped editable installs are not working as they used do:

https://setuptools.pypa.io/en/latest/userguide/development_mode.html#strict-editable-installs

I have been dealing with troubles deriving from this changes. Just pointing out some direction here for future users that might want to try to do a non-editable installs if they haven an issue.

[taningh86]

For sure @zm711 and @h-mayorquin. Sorry I somehow missed the question. So I uninstalled it using pip uninstall spikeinterface in the conda terminal. Then I git cloned it from the source again and did pip install -e .. So that worked. For version 0.101.0 to work a new notebook has to be opened and any other notebooks that were working under the older 0.100.0 won't upgrade to 0.101.0 even after restarting the kernel. Let me know if this is helpful.

[h-mayorquin]

Thanks. I will add this to my first intervention above just so if someone jumps in they can quickly try this.

[taningh86]

I saw that you were using editable installs. Be warned that after setuptools was dropped editable installs are not working as they used do:

https://setuptools.pypa.io/en/latest/userguide/development_mode.html#strict-editable-installs

I have been dealing with troubles deriving from this changes. Just pointing out some direction here for future users that might want to try to do a non-editable installs if they haven an issue.

Oh ok. So should I go ahead and uninstall spikeinterface, git clone from the source and do pip install -e . --config-settings editable_mode=strict to avoid any issues?

[h-mayorquin]

I am actally not sure, maybe something with the kernel of your jupyter notebooks not linking to the latest version of the format. The first thing that I would do in do a normal (non editable) installation and see if that fixes. I was mentioning the problem with the different types of editable installation as that might come up for some people and that's another thing to try.

[zm711]

I think this is now closed with this discussion, but reopen if needed @h-mayorquin.