Sort template to unit and prominent electrode

[RoscoeOBrian]

Hello, I'm using KiloSort and the Phy-GUI on my 6x10 multielectrode data. In the end I get a template.npy file, a template_ind.npy file and a spike_templates.npy file. They provide me with the template waveforms. But I dont now how to link the templates to the units I get out of cluster_groups.csv file. What I want is basically a figure like you see it in the Phy-GUI during the manual sorting. During the sorting process each unit is assined to one electrode, where it occurs most prominently.

Where can I find the assignment Unit - prominent electrode - template waveform (possibly mean waveform)?

[rossant]

You may want to use the template controller which has a get_template_waveforms method. It returns the electrodes and template waveforms.

[RoscoeOBrian]

Thank you for the response. How can I start the template controller? Or is it part of the template-gui? I can not find it there. Perhaps I'm running an outdates version, tho.

[rossant]

Yes first you need to install the latest version, for example by cloning the repo and doing python setup.py develop in it.

Then, you can open IPython or create a Python script with:

from phy.utils._misc import _read_python
from phycontrib.template import TemplateController
c = TemplateController(**_read_python('params.py'))
print(c._get_template_waveforms(3))

[RoscoeOBrian]

Good Morning, I created the Python script (named it "template_controller.py") you provided and saved it in the folder with the params.py file. Then I mapped the location in my windows console and tried to start "template_controller.py". I got the following error:

(phy) Z:location with params.py file>python template_controller.py QWidget: Must construct a QApplication before a QPaintDevice

Is this a known error? Is there something I did wrong?

[rossant]

sorry, you may have to put this at the beginning of the file:

from phy.gui.qt import create_app
create_app()

[RoscoeOBrian]

ok, that made a difference. now I get the error:

10:34:03 [E] cli:39 An error has occured : 'TemplateController' object has no attribute '_get_template_waveforms'

Does that mean, that the "TemplateController" function can not be applied to my params.py file because of the data?

[rossant]

that may be a problem of version, try

pip install git+https://github.com/kwikteam/phy --upgrade
pip install git+https://github.com/kwikteam/phy-contrib --upgrade

[RoscoeOBrian]

Hm ... both are up to date ... and when I take a closer look at gui.py, where TemplateController is included I definitely see the line: def _get_template_waveforms(self, cluster_id) Perhaps it is a problem of the mapped location, so that gui.py can not be found? What location do I have to map, when I run the code that you suggested? The folder where the params.py file is? Or some other location and I missed to implement the params.py file location somewhere in the code you provided?

anyway big thanks for your help!

[rossant]

Weird... have you activated the phy environment before installing phy/phycontrib? Can you check the location of gui.py by doing python -c "import phycontrib; print(phycontrib.__file__)" and check the gui.py at that location?

[RoscoeOBrian]

Yes, it is there. It sais

from .gui import KwikGUIPlugin, KwikController # noga File "C:\Anaconda3\lib\site-packages\phycontrib\kwik\gui.py", line 18, in

[rossant]

That's the problem I think, this is not the phy environment. Do activate phy and then do the pip commands above and then run your script.

[RoscoeOBrian]

I'm sorry for bothering you. I installed everything like you told me and now I get the following error .... (Is it possible, that I installed phycontrib in the phy environment AND somewhere else, and this leads to Problems?)

(phy) Z:\MEA\Gruppe_2\2017_01_09_41_35\h5_Kilosort>template_controller.py
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Z:\MEA\Gruppe_2\2017_01_09_41_35\h5_Kilosort\template_controller.py in <module>(
)
      9
     10 from phy.utils._misc import _read_python
---> 11 from phycontrib.template import TemplateController
     12 c = TemplateController(**_read_python('params.py'))
     13 print(c._get_template_waveforms(3))

C:\Anaconda3\lib\site-packages\phycontrib\__init__.py in <module>()
     13 import shutil
     14
---> 15 from .kwik import KwikGUIPlugin, KwikController  # noqa
     16 from .template import TemplateGUIPlugin, TemplateController  # noqa
     17 from . import utils  # noqa

C:\Anaconda3\lib\site-packages\phycontrib\kwik\__init__.py in <module>()
      8 #-----------------------------------------------------------------------
-------
      9
---> 10 from .gui import KwikGUIPlugin, KwikController  # noqa

C:\Anaconda3\lib\site-packages\phycontrib\kwik\gui.py in <module>()
     16 import numpy as np
     17
---> 18 from phy.cluster.supervisor import Supervisor
     19 from phy.cluster.views import (WaveformView,
     20                                FeatureView,

C:\Anaconda3\lib\site-packages\phy\cluster\__init__.py in <module>()
      7 from .clustering import Clustering
      8 from .supervisor import Supervisor
----> 9 from .views import WaveformView, TraceView, FeatureView, CorrelogramView

C:\Anaconda3\lib\site-packages\phy\cluster\views\__init__.py in <module>()
      8 # ----------------------------------------------------------------------
-------
      9
---> 10 from .correlogram import CorrelogramView  # noqa
     11 from .feature import FeatureView  # noqa
     12 from .scatter import ScatterView  # noqa

C:\Anaconda3\lib\site-packages\phy\cluster\views\correlogram.py in <module>()
     14 from phy.utils import Bunch
     15 from phy.utils._color import _spike_colors
---> 16 from .base import ManualClusteringView
     17
     18 logger = logging.getLogger(__name__)

C:\Anaconda3\lib\site-packages\phy\cluster\views\base.py in <module>()
     10 import logging
     11
---> 12 from vispy.util.event import Event
     13
     14 from phy.gui import Actions

ImportError: No module named 'vispy'

[rossant]

While phy is still activated, do "pip install vispy" and try again. Do the same thing if there are other unavailable modules until it works!

[RoscoeOBrian]

I think I got the Problem: I have two folders:

C:\Anaconda3\envs\phy\Lib\site-packages and C:\Anaconda3\Lib\site-packages

I installed a couple of modules that where missing in the ladder folder and now I get an "Keyerror 3"

KeyError                                  Traceback (most recent call last)
Z:\MEA\Gruppe_2\2017_01_09_41_35\h5_Kilosort\template_controller.py in <module>(
)
     11 from phycontrib.template import TemplateController
     12 c = TemplateController(**_read_python('params.py'))
---> 13 print(c._get_template_waveforms(3))

C:\Anaconda3\lib\site-packages\joblib\memory.py in __call__(self, *args, **kwarg
s)
    481
    482     def __call__(self, *args, **kwargs):
--> 483         return self._cached_call(args, kwargs)[0]
    484
    485     def __reduce__(self):

C:\Anaconda3\lib\site-packages\joblib\memory.py in _cached_call(self, args, kwar
gs)
    428                           'directory %s'
    429                         % (name, argument_hash, output_dir))
--> 430             out, metadata = self.call(*args, **kwargs)
    431             if self.mmap_mode is not None:
    432                 # Memmap the output at the first call to be consistent w
ith

C:\Anaconda3\lib\site-packages\joblib\memory.py in call(self, *args, **kwargs)
    673         if self._verbose > 0:
    674             print(format_call(self.func, args, kwargs))
--> 675         output = self.func(*args, **kwargs)
    676         self._persist_output(output, output_dir)
    677         duration = time.time() - start_time

C:\Anaconda3\lib\site-packages\phycontrib\template\gui.py in _get_template_wavef
orms(self, cluster_id)
    255         """Return the waveforms of the templates corresponding to a clus
ter."""
    256         pos = self.model.channel_positions
--> 257         count = self.get_template_counts(cluster_id)
    258         template_ids = np.nonzero(count)[0]
    259         count = count[template_ids]

C:\Anaconda3\lib\site-packages\phy\io\context.py in memcached(*args)
    116             out = cache.get(h, None)
    117             if out is None:
--> 118                 out = f(*args)
    119                 cache[h] = out
    120             return out

C:\Anaconda3\lib\site-packages\phycontrib\template\gui.py in get_template_counts
(self, cluster_id)
    182         """Return a histogram of the number of spikes in each template f
or
    183         a given cluster."""
--> 184         spike_ids = self.supervisor.clustering.spikes_per_cluster[cluste
r_id]
    185         st = self.model.spike_templates[spike_ids]
    186         return np.bincount(st, minlength=self.model.n_templates)

KeyError: 3

[rossant]

In the script I used "3" as an example of a cluster number. You should change it to the cluster you want to see!

[RoscoeOBrian]

Ha! It worked! I see some data in the console! Now I need to figure out how to run this for all the existing clusters and save it in some sort of file. But I think I get this!

Thanks a lot!!!

[RoscoeOBrian]

Hallo @rossant , I have a followup question. I managed to get the data for all cluster and print them to file. For those who are interested:

loadfile="cluster_groups.csv"
table=open(loadfile,'r')
unitnum=[]
n=0
for line in table:
    if n>0:
        col=line.split()
        #print(col[0])
        unitnum.append(int(col[0]))
    n+=1

numUnits=len(unitnum)

from phy.gui.qt import create_app
create_app()

from phy.utils._misc import _read_python
from phycontrib.template import TemplateController
c = TemplateController(**_read_python('params.py'))
for i in range(numUnits):
    fname='outputfile_unit%d.dat' % (unitnum[i])
    savefile=open(fname,'w')
    savefile.write (str(c._get_template_waveforms(unitnum[i])))
    savefile.close()

I also find the electrode numbers in line 27. It seems, that the first electrode is always the one with the most prominent template. But where can I find the data for the actual template for this electrode? Is the median waveform also saved somewhere?

outputfile_unit4.zip

[rossant]

The _get_template_waveforms() method returns an object with the template itself and the electrodes. The electrodes are sorted by decreasing amplitude, and the waveform array is sorted in the same order.

The mean waveform is available in _get_mean_waveforms(): https://github.com/kwikteam/phy-contrib/blob/master/phycontrib/template/gui.py#L248