Error loading internal MIME data same as issue 1212 - but unresolved

[kipkeller]

Using the sample data with kilosort3, I successfully completed kilosort and can partially load up phy2, but then encounter: 15:14:55.265 [W] model:603 Unreferenced clusters found in templates (generally not a problem) 15:14:55.291 [W] model:625 Unreferenced clusters found in spike_clusters (generally not a problem) 15:14:55.309 [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed. QMimeDatabase: Error loading internal MIME data An error has been encountered at line 1 of : Premature end of document.:

I have done as suggested in 1212 (made a new environment and loaded up the developer version of phy2), but see no change in the issue. I did not specify and versions and just went along with those called for in the install.

thanks

[zm711]

Could we start with which OS are you using and a list of the packages in your phy2 env?

I tend to recommend using the yml approach because it forces you to install everything at once, which helps things play together. You can conda remove the phy2 env (let me know if you need help) and then try downloading the environment.yml file and typing

conda env create -f environment.yml

EDITED: for mixed up env and create

[kipkeller]

Hi Zach, Thank you. Windows 10 Pro

conda list: abseil-cpp 20211102.0 hd77b12b_0 appdirs 1.4.4 pyhd3eb1b0_0 arrow-cpp 11.0.0 py311h308b814_0 asttokens 2.0.5 pyhd3eb1b0_0 attrs 22.1.0 py311haa95532_0 aws-c-common 0.4.57 ha925a31_1 aws-c-event-stream 0.1.6 hd77b12b_5 aws-checksums 0.1.9 ha925a31_0 aws-sdk-cpp 1.8.185 hd77b12b_0 backcall 0.2.0 pyhd3eb1b0_0 blas 1.0 mkl bokeh 3.2.0 py311h746a85d_0 boost-cpp 1.73.0 h2bbff1b_12 bottleneck 1.3.5 py311h5bb9823_0 brotli 1.0.9 h2bbff1b_7 brotli-bin 1.0.9 h2bbff1b_7 brotlipy 0.7.0 py311h2bbff1b_1002 bzip2 1.0.8 he774522_0 c-ares 1.19.0 h2bbff1b_0 ca-certificates 2023.05.30 haa95532_0 certifi 2023.7.22 py311haa95532_0 cffi 1.15.1 py311h2bbff1b_3 charset-normalizer 2.0.4 pyhd3eb1b0_0 click 8.0.4 py311haa95532_0 cloudpickle 2.2.1 py311haa95532_0 colorama 0.4.6 py311haa95532_0 colorcet 3.0.1 pypi_0 pypi comm 0.1.2 py311haa95532_0 contourpy 1.0.5 py311h59b6b97_0 cryptography 41.0.2 py311h31511bf_0 cycler 0.11.0 pyhd3eb1b0_0 cython 3.0.0 py311h2bbff1b_0 cytoolz 0.12.0 py311h2bbff1b_0 dask 2023.6.0 py311haa95532_0 dask-core 2023.6.0 py311haa95532_0 debugpy 1.5.1 py311hd77b12b_0 decorator 5.1.1 pyhd3eb1b0_0 distributed 2023.6.0 py311haa95532_0 executing 0.8.3 pyhd3eb1b0_0 fonttools 4.25.0 pyhd3eb1b0_0 freetype 2.12.1 ha860e81_0 fsspec 2023.4.0 py311haa95532_0 gflags 2.2.2 ha925a31_0 ghp-import 2.1.0 pypi_0 pypi giflib 5.2.1 h8cc25b3_3 glib 2.69.1 h5dc1a3c_2 glog 0.5.0 hd77b12b_0 gst-plugins-base 1.18.5 h9e645db_0 gstreamer 1.18.5 hd78058f_0 h5py 3.7.0 py311h259cc0e_0 hdf5 1.10.6 h1756f20_1 heapdict 1.0.1 pyhd3eb1b0_0 icc_rt 2022.1.0 h6049295_2 icu 58.2 ha925a31_3 idna 3.4 py311haa95532_0 importlib-metadata 6.0.0 py311haa95532_0 iniconfig 1.1.1 pyhd3eb1b0_0 intel-openmp 2023.1.0 h59b6b97_46319 ipykernel 6.19.2 py311h86cfffd_0 ipython 8.12.0 py311haa95532_0 ipython_genutils 0.2.0 pyhd3eb1b0_1 jedi 0.18.1 py311haa95532_1 jinja2 3.1.2 py311haa95532_0 joblib 1.2.0 py311haa95532_0 jpeg 9e h2bbff1b_1 jupyter_client 8.1.0 py311haa95532_0 jupyter_core 5.3.0 py311haa95532_0 kiwisolver 1.4.4 py311hd77b12b_0 klusta 3.0.16 pypi_0 pypi klustakwik2 0.2.6 pypi_0 pypi krb5 1.19.4 h5b6d351_0 lerc 3.0 hd77b12b_0 libboost 1.73.0 h6c2663c_12 libbrotlicommon 1.0.9 h2bbff1b_7 libbrotlidec 1.0.9 h2bbff1b_7 libbrotlienc 1.0.9 h2bbff1b_7 libclang 14.0.6 default_hb5a9fac_1 libclang13 14.0.6 default_h8e68704_1 libcurl 8.1.1 h86230a5_1 libdeflate 1.17 h2bbff1b_0 libevent 2.1.12 hcc03200_0 libffi 3.4.4 hd77b12b_0 libiconv 1.16 h2bbff1b_2 libogg 1.3.5 h2bbff1b_1 libpng 1.6.39 h8cc25b3_0 libprotobuf 3.20.3 h23ce68f_0 libsodium 1.0.18 h62dcd97_0 libssh2 1.10.0 hcd4344a_2 libthrift 0.15.0 he49ee6e_2 libtiff 4.5.0 h6c2663c_2 libvorbis 1.3.7 he774522_0 libwebp 1.2.4 hbc33d0d_1 libwebp-base 1.2.4 h2bbff1b_1 libxml2 2.10.3 h0ad7f3c_0 libxslt 1.1.37 h2bbff1b_0 locket 1.0.0 py311haa95532_0 lz4 4.3.2 py311h2bbff1b_0 lz4-c 1.9.4 h2bbff1b_0 markdown 3.4.4 pypi_0 pypi markupsafe 2.1.1 py311h2bbff1b_0 matplotlib 3.7.1 py311haa95532_1 matplotlib-base 3.7.1 py311hf62ec03_1 matplotlib-inline 0.1.6 py311haa95532_0 mergedeep 1.3.4 pypi_0 pypi mkdocs 1.5.0 pypi_0 pypi mkl 2023.1.0 h8bd8f75_46356 mkl-service 2.4.0 py311h2bbff1b_1 mkl_fft 1.3.6 py311hf62ec03_1 mkl_random 1.2.2 py311hf62ec03_1 msgpack-python 1.0.3 py311h59b6b97_0 mtscomp 1.0.2 pypi_0 pypi munkres 1.1.4 py_0 nest-asyncio 1.5.6 py311haa95532_0 numexpr 2.8.4 py311h1fcbade_1 numpy 1.25.0 py311hdab7c0b_0 numpy-base 1.25.0 py311hd01c5d8_0 openssl 1.1.1u h2bbff1b_0 orc 1.7.4 h623e30f_1 packaging 23.0 py311haa95532_0 pandas 1.5.3 py311heda8569_0 param 1.13.0 pypi_0 pypi parso 0.8.3 pyhd3eb1b0_0 partd 1.2.0 pyhd3eb1b0_1 pathspec 0.11.1 pypi_0 pypi pcre 8.45 hd77b12b_0 phy 2.0b5 pypi_0 pypi phylib 2.4.3 pypi_0 pypi pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 9.4.0 py311hd77b12b_0 pip 23.2.1 py311haa95532_0 platformdirs 2.5.2 py311haa95532_0 pluggy 1.0.0 py311haa95532_1 ply 3.11 py311haa95532_0 pooch 1.4.0 pyhd3eb1b0_0 prompt-toolkit 3.0.36 py311haa95532_0 psutil 5.9.0 py311h2bbff1b_0 pure_eval 0.2.2 pyhd3eb1b0_0 pyarrow 11.0.0 py311h8a3a540_0 pycparser 2.21 pyhd3eb1b0_0 pyct 0.5.0 pypi_0 pypi pygments 2.15.1 py311haa95532_1 pyopengl 3.1.1a1 py311haa95532_0 pyopenssl 23.2.0 py311haa95532_0 pyparsing 3.0.9 py311haa95532_0 pyqt 5.15.7 py311hd77b12b_0 pyqt5-sip 12.11.0 py311hd77b12b_0 pyqtwebengine 5.15.7 py311hd77b12b_0 pysocks 1.7.1 py311haa95532_0 pytest 7.3.1 py311haa95532_0 python 3.11.4 h966fe2a_0 python-dateutil 2.8.2 pyhd3eb1b0_0 python-lmdb 1.4.1 py311hd77b12b_0 pytz 2022.7 py311haa95532_0 pywin32 305 py311h2bbff1b_0 pyyaml 6.0 py311h2bbff1b_1 pyyaml-env-tag 0.1 pypi_0 pypi pyzmq 25.1.0 py311hd77b12b_0 qt-main 5.15.2 he8e5bd7_8 qt-webengine 5.15.9 hb9a9bb5_5 qtconsole 5.4.2 py311haa95532_0 qtpy 2.2.0 py311haa95532_0 qtwebkit 5.212 h2bbfb41_5 re2 2022.04.01 hd77b12b_0 requests 2.31.0 py311haa95532_0 responses 0.13.3 pyhd3eb1b0_0 scikit-learn 1.2.2 py311hd77b12b_1 scipy 1.10.1 py311hc1ccb85_1 setuptools 68.0.0 py311haa95532_0 sip 6.6.2 py311hd77b12b_0 six 1.16.0 pyhd3eb1b0_1 snappy 1.1.9 h6c2663c_0 sortedcontainers 2.4.0 pyhd3eb1b0_0 sqlite 3.41.2 h2bbff1b_0 stack_data 0.2.0 pyhd3eb1b0_0 tbb 2021.8.0 h59b6b97_0 tblib 1.7.0 pyhd3eb1b0_0 threadpoolctl 2.2.0 pyh0d69192_0 tk 8.6.12 h2bbff1b_0 toml 0.10.2 pyhd3eb1b0_0 tomli 2.0.1 py311haa95532_0 toolz 0.12.0 py311haa95532_0 tornado 6.2 py311h2bbff1b_0 tqdm 4.65.0 pypi_0 pypi traitlets 5.7.1 py311haa95532_0 tzdata 2023c h04d1e81_0 urllib3 1.26.16 py311haa95532_0 utf8proc 2.6.1 h2bbff1b_0 vc 14.2 h21ff451_1 vs2015_runtime 14.27.29016 h5e58377_2 watchdog 3.0.0 pypi_0 pypi wcwidth 0.2.5 pyhd3eb1b0_0 wheel 0.38.4 py311haa95532_0 win_inet_pton 1.1.0 py311haa95532_0 xyzservices 2022.9.0 py311haa95532_1 xz 5.4.2 h8cc25b3_0 yaml 0.2.5 he774522_0 zeromq 4.3.4 hd77b12b_0 zict 2.2.0 py311haa95532_0 zipp 3.11.0 py311haa95532_0 zlib 1.2.13 h8cc25b3_0 zstd 1.5.5 hd43e919_0

I will try the yml file tomorrow when I have better access (I used the conda install commands as per the instructions). RIght now the machine is busy with other tasks. Thanks, Kip

---

From: Zach McKenzie @.> Sent: Thursday, August 17, 2023 3:41 PM To: cortex-lab/phy @.> Cc: kipkeller @.>; Author @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Could we start with which OS are you using and a list of the packages in your phy2 env?

I tend to recommend using the yml approach because it forces you to install everything at once, which helps things play together. You can conda remove the phy2 env (let me know if you need help) and then try downloading the environment.yml file and typing

conda create env -f environment.yml

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1683070944, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWAVHPWJLVJBBBDJNJTXV2M3LANCNFSM6AAAAAA3UWTV2I. You are receiving this because you authored the thread.Message ID: @.***>

[zm711]

@kipkeller, Hey Kip, I already see part of the problem. You installed phy from pypi, but there hasn't been an update for the new packaging yet. So you need to build phy from git to make sure it works with the latest python packages. Definitely just do the yaml install. (Windows doesn't come with git preinstalled, so trying to build with pip+git won't work unless you install git into your conda env-- the environment.yml will install git for you automatically). The one issue you can run into is the yaml will name the environment phy2, so you need to remove/delete the old phy2 env or edit the name of the env at the top of the yaml to be something else (phy1, phy3, whatever you want). Cheers, Zach

[zm711]

Also you are installing this into an environment right? It looks like this may be in a base install.

[kipkeller]

Zach,

First, I originally did this (as per https://github.com/cortex-lab/phy#developer-instructions readme file): conda create -n phy2 -y cython dask h5py joblib matplotlib numpy pillow pip pyopengl pyqt pyqtwebengine pytest python qtconsole requests responses scikit-learn scipy traitlets

This created the phy2 environment that I had been using but that gave me the error that brought me to you.

I have now removed phy2 and, from within my phy-contrib folder, I have run: conda env create -f environment.yml (note that you had it as conda create env, but conda env create is correct).

This gave me all sorts of conflicts and ultimately failed (I also note that the yml file would have the env named as phy - not phy2, so I wonder if this is the correct yml - etc. ). This is the phy-contrib folder that was created by the original conda create -n phy2 ... command.

not sure what to try next. Thanks Kip

---

From: Zach McKenzie @.> Sent: Friday, August 18, 2023 4:28 AM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Also you are installing this into an environment right? It looks like this may be in a base install.

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1683780646, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWA6WZXM4Q7SWGAE4WLXV5GWBANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

Howdy Kip (@kipkeller )

first sorry I always get the order mixed up. You're right you should do conda env create.

That is the wrong yml. Grab this exact file environment.yml instead. To do that just right click the blue link and click save as and then save as environment.yml.

This will be in your downloads folder. In your anaconda prompt type:

cd downloads
conda env create -f environment.yml

This will create an environment called phy2. You should look inside the yml and see this.

name: phy2
channels:
  - conda-forge
  - defaults
dependencies:
  - python
  - pip
  - git
  - numpy
  - matplotlib
  - scipy
  - h5py
  - pyqt
  - pyopengl
  - pyqtwebengine
  - pytest
  - qtconsole
  - requests
  - responses
  - traitlets
  - dask
  - cython
  - pillow
  - scikit-learn
  - joblib
  - pip:
    - git+https://github.com/cortex-lab/phy.git

Zach

[kipkeller]

Yes! I had just gotten to this yaml by re-cloning https://github.com/cortex-lab/phy.git and this was the same as the one you linked to me. It seems to work - thanks!!!

Now to try to get kilosort3 to work with our tetrode continuous data from the new open ephys - still having issues there.

thanks so much kip

---

From: Zach McKenzie @.> Sent: Friday, August 18, 2023 9:24 AM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Howdy Kip @.***https://github.com/kipkeller )

first sorry I always get the order mixed up. You're right you should do conda env create.

That is the wrong yml. Grab this exact file environment.yamlhttps://github.com/cortex-lab/phy/blob/master/environment.yml instead. To do that just right click the blue link and click save as and then save as environment.yml.

This will be in your downloads folder. In your anaconda prompt type:

cd downloads conda env create -f environment.yml

This will create an environment called phy2. You should look inside the yml and see this.

name: phy2 channels:

• conda-forge
• defaults dependencies:
• python
• pip
• git
• numpy
• matplotlib
• scipy
• h5py
• pyqt
• pyopengl
• pyqtwebengine
• pytest
• qtconsole
• requests
• responses
• traitlets
• dask
• cython
• pillow
• scikit-learn
• joblib
• pip:
  • git+https://github.com/cortex-lab/phy.git

Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1684151904, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWA45KD2NDSQEUYBQVDXV6JKNANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

@kipkeller That's awesome! Happy that worked. If you're really having trouble getting a file format to play nicely there is a python package trying to help with that: spikeinterface They help with loading file data (as well as running the spike sorter so it is easy to try Kilosort2, 2.5, 3 or other sorters). Otherwise if you just need help with loading the file format the project neo may be helpful to you. I know the Kilosort docs say that for low counts that Kilosort2 may work better, but if your issue is in prepping open ephys data for use with Kilosort then spikeinterface or Neo would probably help guide you.

Hope that helps!

Best, Zach

[kipkeller]

Hi Zach, Thanks for all your help - I wonder if you also have insight into my current KS3/Phy2 problem. Apparently, this is a known issue (e.g. #429), but unresolved. I am using KS3 and PHY2 and originally could not see "FeatureView" in Phy2. This was sort of solved with the suggested workaround of running '> phy extract-waveforms params.py' - but not really. With this, I can see the 'FeatureView' pane, but the cutting feature doesn't work properly (as in the issue #429). From reading the various posts to the issue, it seems this is a KS3 (and KS2.5) problem. Would PHY1 solve this? Can you help with this?

Thanks! Kip Keller

---

From: Zach McKenzie @.> Sent: Friday, August 18, 2023 10:14 AM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

@kipkellerhttps://github.com/kipkeller That's awesome! Happy that worked. If you're really having trouble getting a file format to play nicely there is a python package trying to help with that: spikeinterfacehttps://github.com/SpikeInterface/spikeinterface They help with loading file data (as well as running the spike sorter so it is easy to try Kilosort2, 2.5, 3 or other sorters). Otherwise if you just need help with loading the file format the project neohttps://github.com/NeuralEnsemble/python-neo may be helpful to you. I know the Kilosort docs say that for low counts that Kilosort2 may work better, but if your issue is in prepping open ephys data for use with Kilosort then spikeinterface or Neo would probably help guide you.

Hope that helps!

Best, Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1684207184, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWA7JAOKJOMRCLKYBVTXV6PIJANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

Howdy Kip (@kipkeller),

So based on this: https://github.com/MouseLand/Kilosort/issues/317, it is a problem with Kilosort 3 not with Phy1 or Phy2. Because the clustering is different Marius decided not to allow Kilosort to return the necessary values until he could update them. I believe that his group is now working on Kilosort4, so I would probably guess that this update in Kilosort3 will not be happening.

You have two options, either use Kilosort2 or 2.5 or you could switch over to spikeinterface, which will take care of generating all the necessary values for you after running Kilosort3 by using the export to phy option. My suggestion would be that if you really need to use Kilosort 3 and you can't make 2 or 2.5 work test out spikeinterface and see if it works for you. If you can switch then you can try 2 or 2.5 where Kilosort does write the exact files necessary.

Also I'm going to link the kilosort issue 429 instead of the Phy2 429 just so we trace back to the exact troubleshooting you were doing. (https://github.com/MouseLand/Kilosort/issues/429) Best, Zach

[kipkeller]

Zach, Thanks for this. We have KS3 and now KS25 (the fork that comes from 2.5) running and showing the Featureview in Phy2. BUT, we are confused/concerned by what we see there. The clusters - even for what we think are good spikes are almost all centered at 0,0 and rarely offset. There is almost always just the single cluster at 0,0. In the past (KS1 and Phy1) we would see several clusters, often 'noise' centered at 0,0 and a seemingly good cluster somewhere offset. Shouldn't we be seeing something like this again? If everything is centered at 0,0, there seems no purpose to further cut and split? Are there settings we should change - or are we missing something? Thanks, kip

---

From: Zach McKenzie @.> Sent: Tuesday, August 22, 2023 12:48 PM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Howdy Kip @.***https://github.com/kipkeller),

So based on this: MouseLand/Kilosort#317https://github.com/MouseLand/Kilosort/issues/317, it is a problem with Kilosort 3 not with Phy1 or Phy2. Because the clustering is different Marius decided not to allow Kilosort to return the necessary values until he could update them. I believe that his group is now working on Kilosort4, so I would probably guess that this update in Kilosort3 will not be happening.

You have two options, either use Kilosort2 or 2.5 or you could switch over to spikeinterface, which will take care of generating all the necessary values for you after running Kilosort3 by using the export to phy option. My suggestion would be that if you really need to use Kilosort 3 and you can't make 2 or 2.5 work test out spikeinterface and see if it works for you. If you can switch then you can try 2 or 2.5 where Kilosort does write the exact files necessary.

Also I'm going to link the kilosort issue 429 instead of the Phy2 429 just so we trace back to the exact troubleshooting you were doing. (MouseLand/Kilosort#429https://github.com/MouseLand/Kilosort/issues/429) Best, Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1688829248, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWE6UHE5KHSFJBMSOCDXWUEH7ANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

Hey Kip, I might need to see a screen shot of this. There isn't necessarily a problem with having a cluster centered around 0.

I pulled this image from a Kilosort/Phy training video (sorry for the graininess). But we can see that the "blue" cluster is pretty centered around 0 (close to the other spikes) whereas the red is isolated. That doesn't mean that the "blue" is good or bad you could do other analyses (refractory period, pc metrics like isolation distance, etc, as well as look at waveform, amplitude in the other views) to verify quality. But maybe if we can see what yours is looking like we figure out better what is going on.

A big difference between KS1 and KS2.5/3 is that there is now drift correction occurring during the algorithm. So there might be a bigger change in the PC feature spaces than you're used to. Phy also makes use of the temp_wh.dat which is the preprocessed/drift corrected binary file (so raw clusters that looked "bad" before might look better when drift corrected" rather than the raw binary file (typically a .bin file) that KS1/Phy1 used. So that could also be a difference.

If you're willing to post an example I can at least compare to data I've seen from my work and say whether it looks similar, but I mostly use FeatureView for doing merging and splitting of clusters (regardless of quality) and then when I'm done with Phy I do QC, and then the clusters that pass my post-QC I analyze. Zach

[kipkeller]

Hi Zach, Sorry it has taken me so long to answer. Our problem seems to have miraculously disappeared - and we are not sure how. We don't think we did anything differently, but now the featureview seems ok. One reason for the long delay in responding is that we still are trying to convince ourselves that KS25 is doing a good job of finding spikes. We are trying a few parameter changes - but our understanding of exactly what each one does is marginal at best (despite readthedocs). Pushing onwards. Thanks for your help! Kip

---

From: Zach McKenzie @.> Sent: Tuesday, August 22, 2023 6:21 PM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Hey Kip, I might need to see a screen shot of this. There isn't necessarily a problem with having a cluster centered around 0.

I pulled this image from a Kilosort/Phy training video (sorry for the graininess). But we can see that the "blue" cluster is pretty centered around 0 (close to the other spikes) whereas the red is isolated. That doesn't mean that the "blue" is good or bad you could do other analyses (refractory period, pc metrics like isolation distance, etc, as well as look at waveform, amplitude in the other views) to verify quality. But maybe if we can see what yours is looking like we figure out better what is going on.

[IMG_5676C7336137-1]https://user-images.githubusercontent.com/92116279/262518150-867ce6f8-c313-4615-8c94-138c2578b81e.jpeg

A big difference between KS1 and KS2.5/3 is that there is now drift correction occurring during the algorithm. So there might be a bigger change in the PC feature spaces than you're used to. Phy also makes use of the temp_wh.dat which is the preprocessed/drift corrected binary file (so raw clusters that looked "bad" before might look better when drift corrected" rather than the raw binary file (typically a .bin file) that KS1/Phy1 used. So that could also be a difference.

If you're willing to post an example I can at least compare to data I've seen from my work and say whether it looks similar, but I mostly use FeatureView for doing merging and splitting of clusters (regardless of quality) and then when I'm done with Phy I do QC, and then the clusters that pass my post-QC I analyze. Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1689123624, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWGFCHH433SR5Q7FFBTXWVLITANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

@kipkeller

Great that it's better! Often times Phy only loads a subset of your data to be faster so maybe one time when you opened it the subset of data it opened just happened to look bad, but upon initialization with a new random subset of data it looked much better?

As far as finding spikes have you seen the Kilosort team's latest biorxiv. It goes through a comparison of Kilosort1-4 comparing cases where specific versions of their algorithm are more or less accurate. kilosort.

You could give this a read and try to decide if for your experiments 2, 2.5, or 3 would be better (4 is still unreleased).

Zach

[kipkeller]

Hi Zach, I am really sorry to keep bothering you, but phy2 seems to have broken. It was working well until we had to restart the computer yesterday and this seems to correlate with phy2 no longer working.

??? Thanks !!! Kip

After kilosorting (or also with data previously kilosorted and successfully Phy-ed):

08:20:21.557 [W] model:625 Unreferenced clusters found in spike_clusters (generally not a problem) 08:20:21.697 [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed. Extracting waveforms: 100%|████████████████████████████████████████████████████████████████████████| 1800.966/1800.966 [00:00<00:00, 2322.34it/s]

This last line (Extracting waveforms) is (I think) new.

This creates the three files: _phy_spikes_subset.channels.npy _phy_spikes_subset.spikes.npy _phy_spikes_subset.waveforms.npy

Rerunning phy2 then gives this error: Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\ProgramData\anaconda3\envs\phy2\Scripts\phy.exe__main.py", line 7, in File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\click\core.py", line 1157, in call return self.main(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\click\core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\click\core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\click\core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\click\core.py", line 783, in invoke return callback(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\click\decorators.py", line 33, in new_func return f(get_current_context(), args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phy\apps__init.py", line 245, in template_extract_waveforms model.save_spikes_subset_waveforms( File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phylib\io\model.py", line 1390, in save_spikes_subset_waveforms export_waveforms( File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phylib\io\traces.py", line 658, in export_waveforms writer = NpyWriter(path, shape, dtype) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phylib\io\traces.py", line 559, in init__ self.fp = open(path, 'wb') ^^^^^^^^^^^^^^^^ OSError: [Errno 22] Invalid argument: 'C:\DATA\Spikes25\_phy_spikes_subset.waveforms.npy'

Here is the conda list: #

Name Version Build Channel

asttokens 2.4.0 pyhd8ed1ab_0 conda-forge aws-c-auth 0.7.3 hd125877_3 conda-forge aws-c-cal 0.6.2 hfb91821_0 conda-forge aws-c-common 0.9.0 hcfcfb64_0 conda-forge aws-c-compression 0.2.17 h04c9df6_2 conda-forge aws-c-event-stream 0.3.2 h495bb32_0 conda-forge aws-c-http 0.7.12 h0890e15_1 conda-forge aws-c-io 0.13.32 h83b3346_2 conda-forge aws-c-mqtt 0.9.5 h0fd1aac_1 conda-forge aws-c-s3 0.3.14 h9f49523_3 conda-forge aws-c-sdkutils 0.1.12 h04c9df6_1 conda-forge aws-checksums 0.1.17 h04c9df6_1 conda-forge aws-crt-cpp 0.23.0 hf4e30cf_5 conda-forge aws-sdk-cpp 1.11.156 hf794bbe_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge bokeh 3.2.2 pyhd8ed1ab_0 conda-forge brotli 1.1.0 hcfcfb64_0 conda-forge brotli-bin 1.1.0 hcfcfb64_0 conda-forge brotli-python 1.1.0 py311h12c1d0e_0 conda-forge bzip2 1.0.8 h8ffe710_4 conda-forge c-ares 1.19.1 hcfcfb64_0 conda-forge ca-certificates 2023.7.22 h56e8100_0 conda-forge cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge certifi 2023.7.22 pyhd8ed1ab_0 conda-forge charset-normalizer 3.2.0 pyhd8ed1ab_0 conda-forge click 8.1.7 win_pyh7428d3b_0 conda-forge cloudpickle 2.2.1 pyhd8ed1ab_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge colorcet 3.0.1 pypi_0 pypi comm 0.1.4 pyhd8ed1ab_0 conda-forge contourpy 1.1.0 py311h005e61a_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge cython 3.0.2 py311h12c1d0e_0 conda-forge cytoolz 0.12.2 py311ha68e1ae_0 conda-forge dask 2023.9.1 pyhd8ed1ab_0 conda-forge dask-core 2023.9.1 pyhd8ed1ab_0 conda-forge debugpy 1.6.8 py311h12c1d0e_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge distributed 2023.9.1 pyhd8ed1ab_0 conda-forge exceptiongroup 1.1.3 pyhd8ed1ab_0 conda-forge executing 1.2.0 pyhd8ed1ab_0 conda-forge fonttools 4.42.1 py311ha68e1ae_0 conda-forge freetype 2.12.1 h546665d_1 conda-forge fsspec 2023.9.0 pyh1a96a4e_0 conda-forge gettext 0.21.1 h5728263_0 conda-forge ghp-import 2.1.0 pypi_0 pypi git 2.42.0 h57928b3_0 conda-forge glib 2.76.4 h12be248_0 conda-forge glib-tools 2.76.4 h12be248_0 conda-forge gst-plugins-base 1.22.5 h001b923_1 conda-forge gstreamer 1.22.5 hb4038d2_1 conda-forge h5py 3.9.0 nompi_py311h0d04526_102 conda-forge hdf5 1.14.2 nompi_h73e8ff5_100 conda-forge icu 72.1 h63175ca_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge importlib-metadata 6.8.0 pyha770c72_0 conda-forge importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge iniconfig 2.0.0 pyhd8ed1ab_0 conda-forge intel-openmp 2023.2.0 h57928b3_49496 conda-forge ipykernel 6.25.2 pyh60829e3_0 conda-forge ipython 8.15.0 pyh5737063_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge jedi 0.19.0 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge joblib 1.3.2 pyhd8ed1ab_0 conda-forge jupyter_client 8.3.1 pyhd8ed1ab_0 conda-forge jupyter_core 5.3.1 py311h1ea47a8_0 conda-forge kiwisolver 1.4.5 py311h005e61a_0 conda-forge krb5 1.21.2 heb0366b_0 conda-forge lcms2 2.15 h3e3b177_1 conda-forge lerc 4.0.0 h63175ca_0 conda-forge libabseil 20230125.3 cxx17_h63175ca_0 conda-forge libaec 1.0.6 h63175ca_1 conda-forge libarrow 13.0.0 h1d4cf54_2_cpu conda-forge libblas 3.9.0 18_win64_mkl conda-forge libbrotlicommon 1.1.0 hcfcfb64_0 conda-forge libbrotlidec 1.1.0 hcfcfb64_0 conda-forge libbrotlienc 1.1.0 hcfcfb64_0 conda-forge libcblas 3.9.0 18_win64_mkl conda-forge libclang 15.0.7 default_h77d9078_3 conda-forge libclang13 15.0.7 default_h77d9078_3 conda-forge libcrc32c 1.1.2 h0e60522_0 conda-forge libcurl 8.2.1 hd5e4a3a_0 conda-forge libdeflate 1.18 hcfcfb64_0 conda-forge libevent 2.1.12 h3671451_1 conda-forge libexpat 2.5.0 h63175ca_1 conda-forge libffi 3.4.2 h8ffe710_5 conda-forge libglib 2.76.4 he8f3873_0 conda-forge libgoogle-cloud 2.12.0 hbc1b25b_1 conda-forge libgrpc 1.56.2 hea2d5f7_1 conda-forge libhwloc 2.9.2 default_haede6df_1009 conda-forge libiconv 1.17 h8ffe710_0 conda-forge libjpeg-turbo 2.1.5.1 hcfcfb64_0 conda-forge liblapack 3.9.0 18_win64_mkl conda-forge libogg 1.3.4 h8ffe710_1 conda-forge libpng 1.6.39 h19919ed_0 conda-forge libprotobuf 4.23.3 h1975477_1 conda-forge libsodium 1.0.18 h8d14728_1 conda-forge libsqlite 3.43.0 hcfcfb64_0 conda-forge libssh2 1.11.0 h7dfc565_0 conda-forge libthrift 0.19.0 h06f6336_0 conda-forge libtiff 4.5.1 h6c8260b_1 conda-forge libutf8proc 2.8.0 h82a8f57_0 conda-forge libvorbis 1.3.7 h0e60522_0 conda-forge libwebp 1.3.1 hcfcfb64_0 conda-forge libwebp-base 1.3.1 hcfcfb64_0 conda-forge libxcb 1.15 hcd874cb_0 conda-forge libxml2 2.11.5 hc3477c8_1 conda-forge libzlib 1.2.13 hcfcfb64_5 conda-forge locket 1.0.0 pyhd8ed1ab_0 conda-forge lz4 4.3.2 py311haddf500_0 conda-forge lz4-c 1.9.4 hcfcfb64_0 conda-forge m2w64-gcc-libgfortran 5.3.0 6 conda-forge m2w64-gcc-libs 5.3.0 7 conda-forge m2w64-gcc-libs-core 5.3.0 7 conda-forge m2w64-gmp 6.1.0 2 conda-forge m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge markdown 3.4.4 pypi_0 pypi markupsafe 2.1.3 py311ha68e1ae_0 conda-forge matplotlib 3.7.2 py311h1ea47a8_0 conda-forge matplotlib-base 3.7.2 py311h6e989c2_0 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mergedeep 1.3.4 pypi_0 pypi mkdocs 1.5.2 pypi_0 pypi mkl 2022.1.0 h6a75c08_874 conda-forge msgpack-python 1.0.5 py311h005e61a_0 conda-forge msys2-conda-epoch 20160418 1 conda-forge mtscomp 1.0.2 pypi_0 pypi munkres 1.1.4 pyh9f0ad1d_0 conda-forge nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge numpy 1.25.2 py311h0b4df5a_0 conda-forge openjpeg 2.5.0 ha2aaf27_2 conda-forge openssl 3.1.2 hcfcfb64_0 conda-forge orc 1.9.0 hf2b8f0d_1 conda-forge packaging 23.1 pyhd8ed1ab_0 conda-forge pandas 2.1.0 py311hf63dbb6_0 conda-forge param 1.13.0 pypi_0 pypi parso 0.8.3 pyhd8ed1ab_0 conda-forge partd 1.4.0 pyhd8ed1ab_0 conda-forge pathspec 0.11.2 pypi_0 pypi pcre2 10.40 h17e33f8_0 conda-forge phy 2.0b5 pypi_0 pypi phylib 2.4.3 pypi_0 pypi pickleshare 0.7.5 py_1003 conda-forge pillow 10.0.0 py311hde623f7_0 conda-forge pip 23.2.1 pyhd8ed1ab_0 conda-forge platformdirs 3.10.0 pyhd8ed1ab_0 conda-forge pluggy 1.3.0 pyhd8ed1ab_0 conda-forge ply 3.11 py_1 conda-forge pooch 1.7.0 pyha770c72_3 conda-forge prompt-toolkit 3.0.39 pyha770c72_0 conda-forge prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge psutil 5.9.5 py311ha68e1ae_0 conda-forge pthread-stubs 0.4 hcd874cb_1001 conda-forge pthreads-win32 2.9.1 hfa6e2cd_3 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge pyarrow 13.0.0 py311h6a6099b_2_cpu conda-forge pyct 0.5.0 pypi_0 pypi pygments 2.16.1 pyhd8ed1ab_0 conda-forge pyopengl 3.1.6 pyhd8ed1ab_1 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pyqt 5.15.9 py311h125bc19_4 conda-forge pyqt5-sip 12.12.2 py311h12c1d0e_4 conda-forge pyqtwebengine 5.15.9 py311h5a77453_4 conda-forge pysocks 1.7.1 pyh0701188_6 conda-forge pytest 7.4.1 pyhd8ed1ab_0 conda-forge python 3.11.5 h2628c8c_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge python_abi 3.11 3_cp311 conda-forge pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge pywin32 304 py311h12c1d0e_2 conda-forge pyyaml 6.0.1 py311ha68e1ae_0 conda-forge pyyaml-env-tag 0.1 pypi_0 pypi pyzmq 25.1.1 py311h7b3f143_0 conda-forge qt-main 5.15.8 h063a7da_15 conda-forge qt-webengine 5.15.8 h5b1ea0b_0 conda-forge qtconsole 5.4.4 pyhd8ed1ab_0 conda-forge qtconsole-base 5.4.4 pyha770c72_0 conda-forge qtpy 2.4.0 pyhd8ed1ab_0 conda-forge re2 2023.03.02 hd4eee63_0 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge responses 0.23.2 pyhd8ed1ab_1 conda-forge scikit-learn 1.3.0 py311h142b183_0 conda-forge scipy 1.11.2 py311h37ff6ca_0 conda-forge setuptools 68.1.2 pyhd8ed1ab_0 conda-forge sip 6.7.11 py311h12c1d0e_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.10 hfb803bf_0 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge stack_data 0.6.2 pyhd8ed1ab_0 conda-forge tbb 2021.10.0 h91493d7_0 conda-forge tblib 1.7.0 pyhd8ed1ab_0 conda-forge threadpoolctl 3.2.0 pyha21a80b_0 conda-forge tk 8.6.12 h8ffe710_0 conda-forge toml 0.10.2 pyhd8ed1ab_0 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge toolz 0.12.0 pyhd8ed1ab_0 conda-forge tornado 6.3.3 py311ha68e1ae_0 conda-forge tqdm 4.66.1 pypi_0 pypi traitlets 5.9.0 pyhd8ed1ab_0 conda-forge types-pyyaml 6.0.12.11 pyhd8ed1ab_0 conda-forge typing-extensions 4.7.1 hd8ed1ab_0 conda-forge typing_extensions 4.7.1 pyha770c72_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge ucrt 10.0.22621.0 h57928b3_0 conda-forge urllib3 2.0.4 pyhd8ed1ab_0 conda-forge vc 14.3 h64f974e_17 conda-forge vc14_runtime 14.36.32532 hfdfe4a8_17 conda-forge vs2015_runtime 14.36.32532 h05e6639_17 conda-forge watchdog 3.0.0 pypi_0 pypi wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge wheel 0.41.2 pyhd8ed1ab_0 conda-forge win_inet_pton 1.1.0 pyhd8ed1ab_6 conda-forge xorg-libxau 1.0.11 hcd874cb_0 conda-forge xorg-libxdmcp 1.1.3 hcd874cb_0 conda-forge xyzservices 2023.7.0 pyhd8ed1ab_0 conda-forge xz 5.2.6 h8d14728_0 conda-forge yaml 0.2.5 h8ffe710_2 conda-forge zeromq 4.3.4 h0e60522_1 conda-forge zict 3.0.0 pyhd8ed1ab_0 conda-forge zipp 3.16.2 pyhd8ed1ab_0 conda-forge zstd 1.5.5 h12be248_0 conda-forge

---

From: Zach McKenzie @.> Sent: Tuesday, August 29, 2023 9:54 AM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

@kipkellerhttps://github.com/kipkeller

Great that it's better! Often times Phy only loads a subset of your data to be faster so maybe one time when you opened it the subset of data it opened just happened to look bad, but upon initialization with a new random subset of data it looked much better?

As far as finding spikes have you seen the Kilosort team's latest biorxiv. It goes through a comparison of Kilosort1-4 comparing cases where specific versions of their algorithm are more or less accurate. kilosorthttps://www.biorxiv.org/content/10.1101/2023.01.07.523036v1.

You could give this a read and try to decide if for your experiments 2, 2.5, or 3 would be better (4 is still unreleased).

Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1697815742, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWDA2YCR7QUCQ33EAD3XXYNDLANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

@kipkeller No worries at all. Couple questions to start. Do you still have the raw binary file around .bin or .dat file? What command did you run? (phy template-gui params.py ?) Where is this? OSError: [Errno 22] Invalid argument: 'C:\\DATA\\Spikes25\\_phy_spikes_subset.waveforms.npy' coming from. I might need to see your rough file structure (for example my data is in C:\users\kip\my_spike_sorting).

Finally could you run phy with the --debug so we can get everything that is going on so we can try to see where in the phy world the error is occurring. So run:

phy template-gui params.py --debug and just copy the whole thing here so we can see each step and try to isolate which step is failing.

[kipkeller]

OK ... magic. Running it in the debug mode magically fixed it and I can't get it to error again. This after many attempts, re-installing, etc. I guess I am happy - but ...

Thanks for this! kip

---

From: Zach McKenzie @.> Sent: Thursday, September 7, 2023 10:36 AM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

@kipkellerhttps://github.com/kipkeller No worries at all. Couple questions to start. Do you still have the raw binary file around .bin or .dat file? What command did you run? (phy template-gui params.py ?) Where is this? OSError: [Errno 22] Invalid argument: 'C:\DATA\Spikes25\_phy_spikes_subset.waveforms.npy' coming from. I might need to see your rough file structure (for example my data is in C:\users\kip\my_spike_sorting).

Finally could you run phy with the --debug so we can get everything that is going on so we can try to see where in the phy world the error is occurring. So run:

phy template-gui params.py --debug and just copy the whole thing here so we can see each step and try to isolate which step is failing.

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1710540327, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWGMHN3GXNJM52FZAITXZIAY5ANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

@kipkeller

I'll take magic any day of the week (unless it's on my own computer haha). Zach

[kipkeller]

Hi Zach, Me again. Sorry. So, we are using KS2.0 (because we have tetrodes and 2.0 seems to work better for tetrodes). And we are using Phy2. However, we are unable to see all the spikes (we see only the mean) in "waveform view." Toggling 'M' doesn't help. I think I have (sort of) tracked this down to the params.py file created in rezToPhy. In KS2.0 this sets the dat_path to rez.ops.binary (which is the input file and was in a separate folder): [~, fname, ext] = fileparts(rez.ops.fbinary); fprintf(fid,['dat_path = ''',fname ext '''\n']);

In KS2.5 it sets dat_path to rez.ops.fproc, which appears to be the whitened file ("temp_wh.dat" and does exist in the output folder)

I have tried each (after copying the input file to the output folder), and each provides waveforms to the waveform view, but they don't look correct (e.g. using the input data, the waveforms seem to have identical large 'spikes' on all channels...).

Seems like the temp_wh.dat should be correct, but the true spikes go away and other high frequency stuff is there instead

With neither file present, the means look good - but there is no data to (individual waveforms) to work with.

Does any of this make sense? Can you suggest a fix?

Thanks, kip

---

From: Zach McKenzie @.> Sent: Thursday, September 7, 2023 11:09 AM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

@kipkellerhttps://github.com/kipkeller

I'll take magic any day of the week (unless it's on my own computer haha). Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1710579053, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWCN6WDJ3VSACXR2KTDXZIEW5ANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[zm711]

Howdy Kip,

That sounds frustrating. So the *.bin vs the temp_wh.dat is a debate I've recently had. Basically the temp_wh is as you said whitened and has preprocessing (high pass filter etc). So the data that Kilosort is sorting on is always the temp_wh. The switch in using bin to using temp_wh actually occurred with the drift correction of KS2.5+. This is because it would be wrong to try to look at the raw files of drifting neurons, since Kilosort is already correcting for the drift. Since Kilosort 2 does drift tracking instead of correction it is unclear if you absolutely need to view the waveforms after preprocessing or whether the completely raw is okay.

If you decide to look at the raw then you would need to add the *.bin file to the output folder. That is required so you did that right.

As far as the noise you're seeing there are two options:

It could be real, which means you have to figure out why your tetrode recording is noisy (ie grounding etc). It could be display artifact. I've had this before (I don't know why). Usually shutting down the computer and restarting everything solves it for me. But I would also recommend doing a --debug just in case. So if it displays an error it might let you know of a problem with it reading the data.

Does that make sense?

Zach

[kipkeller]

Hi Zack, Thanks for your help. I follow, except for when you talk about .bin. The input data to kilosort is a .mat file (MATLAB format). Should there be a .bin as well? And what is the binary format? I do vaguely remember a .bin from a while back, but we don't get one now. And I don't know how it might have been generated. I am about to try --debug and shutting down and will let you know how they work. Thanks! kip

---

From: Zach McKenzie @.> Sent: Tuesday, September 12, 2023 4:11 PM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Howdy Kip,

That sounds frustrating. So the *.bin vs the temp_wh.dat is a debate I've recently had. Basically the temp_wh is as you said whitened and has preprocessing (high pass filter etc). So the data that Kilosort is sorting on is always the temp_wh. The switch in using bin to using temp_wh actually occurred with the drift correction of KS2.5+. This is because it would be wrong to try to look at the raw files of drifting neurons, since Kilosort is already correcting for the drift. Since Kilosort 2 does drift tracking instead of correction it is unclear if you absolutely need to view the waveforms after preprocessing or whether the completely raw is okay.

If you decide to look at the raw then you would need to add the *.bin file to the output folder. That is required so you did that right.

As far as the noise you're seeing there are two options:

It could be real, which means you have to figure out why your tetrode recording is noisy (ie grounding etc). It could be display artifact. I've had this before (I don't know why). Usually shutting down the computer and restarting everything solves it for me. But I would also recommend doing a --debug just in case. So if it displays an error it might let you know of a problem with it reading the data.

Does that make sense?

Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1716641540, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWDHH4CTAYAWBISBYALX2DTYVANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[kipkeller]

I am wrong. It's a .dat binary file made in MATLAB. So, I guess I understand that part now. thnx kip

---

From: Zach McKenzie @.> Sent: Tuesday, September 12, 2023 4:11 PM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Howdy Kip,

That sounds frustrating. So the *.bin vs the temp_wh.dat is a debate I've recently had. Basically the temp_wh is as you said whitened and has preprocessing (high pass filter etc). So the data that Kilosort is sorting on is always the temp_wh. The switch in using bin to using temp_wh actually occurred with the drift correction of KS2.5+. This is because it would be wrong to try to look at the raw files of drifting neurons, since Kilosort is already correcting for the drift. Since Kilosort 2 does drift tracking instead of correction it is unclear if you absolutely need to view the waveforms after preprocessing or whether the completely raw is okay.

If you decide to look at the raw then you would need to add the *.bin file to the output folder. That is required so you did that right.

As far as the noise you're seeing there are two options:

It could be real, which means you have to figure out why your tetrode recording is noisy (ie grounding etc). It could be display artifact. I've had this before (I don't know why). Usually shutting down the computer and restarting everything solves it for me. But I would also recommend doing a --debug just in case. So if it displays an error it might let you know of a problem with it reading the data.

Does that make sense?

Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1716641540, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWDHH4CTAYAWBISBYALX2DTYVANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[kipkeller]

Hi Zach,

I have restarted our computer. Here are screen shots of our waveform view under various conditions (note I have set the ops.fshigh=600 as this is our usual openephys high pass filter setting).

without the *.dat file (toggling ''M' has no effect): - probably a nice cell: [cid:e5eba716-a0f4-4bee-912f-4e0bb9eb2303]

with either the *.dat or the temp_wh file: - they give the same: - no response obvious [cid:3f797c23-1032-46bd-aa6c-5b5db19d2f0e]

with either *.dat or temp_wh after toggling 'M': no response obvious [cid:b1013c18-ac20-4a50-891a-fbcfb0c85ac0]

Here is the debug output: phy template-gui params.py --debug 09:16:05.260 [D] init:68 Start capturing exceptions. 09:16:05.291 [W] model:603 Unreferenced clusters found in templates (generally not a problem) 09:16:05.307 [D] model:619 Loading spike clusters. 09:16:05.338 [W] model:625 Unreferenced clusters found in spike_clusters (generally not a problem) 09:16:05.370 [D] model:569 No channel shank file found. 09:16:05.370 [D] model:692 Loading templates. 09:16:05.370 [D] model:724 Templates are dense. 09:16:05.370 [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed. 09:16:05.370 [D] model:730 Loading the whitening matrix. 09:16:05.370 [D] model:737 Loading the inverse of the whitening matrix. 09:16:05.370 [D] model:766 Loading features. 09:16:05.370 [D] model:781 Features are sparse. 09:16:05.370 [D] model:803 Loading template features. 09:16:05.370 [D] model:813 Template features are sparse. 09:16:05.370 [D] model:504 Load cluster_Amplitude.tsv. 09:16:05.370 [D] model:504 Load cluster_ContamPct.tsv. 09:16:05.370 [D] model:504 Load cluster_group.tsv. 09:16:05.370 [D] model:504 Load cluster_KSLabel.tsv. 09:16:05.630 [D] context:100 Initialize joblib cache dir at C:\DATA\Spikes25\.phy. 09:16:05.630 [D] context:101 Reducing the size of the cache if needed. 09:16:05.645 [D] base:102 Add filter high_pass. 09:16:05.645 [D] config:31 Load config file C:\Users\wehr\.phy\phy_config.py. 09:16:05.661 [D] plugin:145 Loading 0 plugins. 09:16:05.661 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms. 09:16:05.661 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms. 09:16:05.661 [D] context:126 Load memcache for phy.apps.base._get_template_waveforms. 09:16:05.661 [D] context:126 Load memcache for phy.apps.base.get_mean_spike_template_amplitudes. 09:16:05.661 [D] context:126 Load memcache for phy.apps.base.get_template_counts. 09:16:05.661 [D] context:126 Load memcache for phy.apps.base.get_template_for_cluster. 09:16:05.661 [D] context:126 Load memcache for phy.apps.template.gui.get_template_amplitude. 09:16:05.676 [D] context:126 Load memcache for phy.apps.base.get_cluster_amplitude. 09:16:05.676 [D] context:126 Load memcache for phy.apps.base.get_mean_firing_rate. 09:16:05.676 [D] context:126 Load memcache for phy.apps.base.get_best_channel. 09:16:05.676 [D] context:126 Load memcache for phy.apps.template.gui.get_best_channels. 09:16:05.676 [D] context:126 Load memcache for phy.apps.base.get_channel_shank. 09:16:05.676 [D] context:126 Load memcache for phy.apps.base.get_probe_depth. 09:16:05.676 [D] context:126 Load memcache for phy.apps.base.peak_channel_similarity. 09:16:05.692 [D] context:185 Save data to C:\DATA\Spikes25\.phy\spikes_per_cluster.pkl. 09:16:05.708 [D] gui:463 Creating GUI. 09:16:05.723 [D] state:46 Load C:\Users\wehr.phy\TemplateGUI\state.json for GUIState. 09:16:05.723 [D] state:46 Load C:\DATA\Spikes25.phy\state.json for GUIState. 09:16:06.068 [D] gui:718 Add view ClusterView to GUI. 09:16:06.087 [D] gui:718 Add view SimilarityView to GUI. 09:16:06.122 [D] gui:718 Add view WaveformView to GUI. 09:16:06.368 [D] base:337 Set state for WaveformView. 09:16:06.379 [D] gui:718 Add view CorrelogramView to GUI. 09:16:06.381 [D] base:337 Set state for CorrelogramView. 09:16:06.400 [D] gui:718 Add view ISIView to GUI. 09:16:06.403 [D] base:337 Set state for ISIView. 09:16:06.414 [D] gui:718 Add view FeatureView to GUI. 09:16:06.416 [D] base:337 Set state for FeatureView. 09:16:06.436 [D] gui:718 Add view AmplitudeView to GUI. 09:16:06.438 [D] base:337 Set state for AmplitudeView. 09:16:06.556 [D] gui:519 Load the geometry state. 09:16:07.063 [D] supervisor:817 Clusters selected: [56] (15) 09:16:09.551 [D] waveform:512 Switch to mean waveforms. 09:16:16.071 [D] waveform:508 Switch to raw waveforms. 09:16:22.821 [D] state:171 Update GUI state for ClusterView 09:16:22.837 [D] state:171 Update GUI state for WaveformView 09:16:22.837 [D] state:171 Update GUI state for CorrelogramView 09:16:22.837 [D] state:171 Update GUI state for ISIView 09:16:22.837 [D] state:171 Update GUI state for FeatureView 09:16:22.837 [D] state:171 Update GUI state for AmplitudeView 09:16:22.837 [D] context:138 Save memcache for phy.apps.base._get_mean_waveforms. 09:16:22.837 [D] context:138 Save memcache for phy.apps.base._get_template_waveforms. 09:16:22.837 [D] context:138 Save memcache for phy.apps.base.get_mean_spike_template_amplitudes. 09:16:22.853 [D] context:138 Save memcache for phy.apps.base.get_template_counts. 09:16:22.853 [D] context:138 Save memcache for phy.apps.base.get_template_for_cluster. 09:16:22.853 [D] context:138 Save memcache for phy.apps.template.gui.get_template_amplitude. 09:16:22.853 [D] context:138 Save memcache for phy.apps.base.get_cluster_amplitude. 09:16:22.853 [D] context:138 Save memcache for phy.apps.base.get_mean_firing_rate. 09:16:22.853 [D] context:138 Save memcache for phy.apps.base.get_best_channel. 09:16:22.853 [D] context:138 Save memcache for phy.apps.template.gui.get_best_channels. 09:16:22.853 [D] context:138 Save memcache for phy.apps.base.get_channel_shank. 09:16:22.853 [D] context:138 Save memcache for phy.apps.base.get_probe_depth. 09:16:22.868 [D] context:138 Save memcache for phy.apps.base.peak_channel_similarity. 09:16:22.868 [D] gui:600 Save the geometry state. 09:16:22.868 [D] state:218 Save global GUI state to C:\Users\wehr\.phy\TemplateGUI\state.json. 09:16:22.868 [D] state:232 Save local GUI state to C:\DATA\Spikes25\.phy\state.json. 09:16:22.901 [D] model:263 Close memmap array features. 09:16:22.901 [D] model:263 Close memmap array sparse_clusters.data. 09:16:22.901 [D] model:263 Close memmap array sparse_features.data. 09:16:22.901 [D] model:263 Close memmap array sparse_features.cols. 09:16:22.901 [D] model:263 Close memmap array sparse_template_features.data. 09:16:22.901 [D] model:263 Close memmap array sparse_templates.data. 09:16:22.901 [D] model:263 Close memmap array template_features. 09:16:22.901 [D] init:87 Stop capturing exceptions.

Any suggestions? - We would really like to use the waveforms and not just the mean. Thanks! kip

---

From: Zach McKenzie @.> Sent: Tuesday, September 12, 2023 4:11 PM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

Howdy Kip,

That sounds frustrating. So the *.bin vs the temp_wh.dat is a debate I've recently had. Basically the temp_wh is as you said whitened and has preprocessing (high pass filter etc). So the data that Kilosort is sorting on is always the temp_wh. The switch in using bin to using temp_wh actually occurred with the drift correction of KS2.5+. This is because it would be wrong to try to look at the raw files of drifting neurons, since Kilosort is already correcting for the drift. Since Kilosort 2 does drift tracking instead of correction it is unclear if you absolutely need to view the waveforms after preprocessing or whether the completely raw is okay.

If you decide to look at the raw then you would need to add the *.bin file to the output folder. That is required so you did that right.

As far as the noise you're seeing there are two options:

It could be real, which means you have to figure out why your tetrode recording is noisy (ie grounding etc). It could be display artifact. I've had this before (I don't know why). Usually shutting down the computer and restarting everything solves it for me. But I would also recommend doing a --debug just in case. So if it displays an error it might let you know of a problem with it reading the data.

Does that make sense?

Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1716641540, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWDHH4CTAYAWBISBYALX2DTYVANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.

[zm711]

@kipkeller

The pictures aren't displaying. Do you want to try to reupload? The snipping tool works fine for uploading an image.

09:16:07.063 [D] supervisor:817 Clusters selected: [56] (15) 09:16:09.551 [D] waveform:512 Switch to mean waveforms. 09:16:16.071 [D] waveform:508 Switch to raw waveforms.

Above indicates that the GUI thinks that it is switching between the mean and raw waveforms, but I thin I would need to see them before I could try to help.

Thanks,

Zach

[kipkeller]

I did use the snipping tool originally. WIll try again here and also attach them. There are three: without_tempData - won't toggle, no file to toggle to With_tempData - using original data (showing means, but no spikes visible) With_temp_wh - using whitened data (showing waveforms, no spikes visible)

The 2nd & 3rd show the same thing using either file for waveforms

[cid:f36ca1ff-42a7-4475-8b57-eedabf12b2bd]

[cid:7c637764-5408-4d40-9390-b95ac5873cdc]

[cid:21dcb9ab-5643-4e75-ad65-50e8c41c6af1]

kip

---

From: Zach McKenzie @.> Sent: Wednesday, September 13, 2023 11:55 AM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

@kipkellerhttps://github.com/kipkeller

The pictures aren't displaying. Do you want to try to reupload? The snipping tool works fine for uploading an image.

09:16:07.063 [D] supervisor:817 Clusters selected: [56] (15) 09:16:09.551 [D] waveform:512 Switch to mean waveforms. 09:16:16.071 [D] waveform:508 Switch to raw waveforms.

Above indicates that the GUI thinks that it is switching between the mean and raw waveforms, but I thin I would need to see them before I could try to help.

Thanks,

Zach

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1718155111, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWEWXFYIGGJU6S2XMADX2H6THANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.

[zm711]

I think it is blocking it because it is going through your email. Zach

[kipkeller]

[zm711]

Again using the Phy tutorial from the Kilosort/Phy people what we would expect from the data would be something like this. Usually the mean waveforms would not appear quite that sharp which makes this seem a bit more like noise. What does the raw trace look like for Phy? So not the waveform view, but the TraceView. This should load the raw data and will highlight what it is calling a spike. If that looks normal there might be a problem with the waveforms, but I agree these do not look like successes to me.

[kipkeller]

This is the same data but a different channel - that is a bit more above the noise floor [cid:5d527783-fbb4-4b82-bd0b-545d7013825c]

---

From: Zach McKenzie @.> Sent: Wednesday, September 13, 2023 12:40 PM To: cortex-lab/phy @.> Cc: kipkeller @.>; Mention @.> Subject: Re: [cortex-lab/phy] Error loading internal MIME data same as issue 1212 - but unresolved (Issue #1218)

[image]https://user-images.githubusercontent.com/92116279/267769140-9900def3-3a0a-40c3-82a8-0d98e7eda847.png

Again using the Phy tutorial from the Kilosort/Phy people what we would expect from the data would be something like this. Usually the mean waveforms would not appear quite that sharp which makes this seem a bit more like noise. What does the raw trace look like for Phy? So not the waveform view, but the TraceView. This should load the raw data and will highlight what it is calling a spike. If that looks normal there might be a problem with the waveforms, but I agree these do not look like successes to me.

— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1218#issuecomment-1718212555, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHZUXWDVC3WIRMNGOFVKHRTX2IDZJANCNFSM6AAAAAA3UWTV2I. You are receiving this because you were mentioned.Message ID: @.***>

[kipkeller]

and for clarification - the more angular plot was the templates (shown on waveform view).

[zm711]

At least for me the SNR still doesn't look super great. I would probably do some analysis outside of phy to look at SNR, and plot some waveforms myself. That would be quite a bit of code to put right here, but it shouldn't be too difficult to plot a random permutation of waveforms yourself and see if when you plot raw waveforms (or from temp_wh) do they look okay. At least based on what I'm seeing I can understand why Phy is struggling generating a mean waveform. The template it is using is what kilosort attempted to use to sort your data, but if the SNR isn't great then Kilosort might overlay some less than ideal raw spikes on the idealized template.

[kipkeller]

Thanks Zach, I agree, these SNRs look poor in general. I picked a different session with better SNRs to come up with some 'fixes', which in the short run seem to help: in KS2.0, it seems the 'update GUI', including being rather slow, may not be removing all traces of the previous usage. To be safe, I just eliminated the code to repopulate with the old settings and now start fresh each time. I also hardcoded in our fshigh=600 (instead of the default 150). In phy2, I permanently moved the input data, *.dat", file into the working directory and changed the params.py file (in rezToPhy) to use that file instead of the whitened file. This seems to be the major game-changer. The data_wh.dat did not work. Any cutting would result in loss of the waveform (resulting in just 'noise' showing in the waveform view, where the spike used to appear). Using the original data file (so far) seems to work as expected. But, given the seemingly random appearance of new problems with KS/phy2, I am waiting for my confidence to build. Thank you for all your help - and especially for your patience! Kip

[zm711]

Kip,

Sounds like its the best you can do. Like I said we were having a discussion on another project about that the other day (temp_wh vs raw file) and for KS2.5 or 3 we decided temp_wh was better, but we weren't sure about KS2. So I'll at least share your experience.

KS is pretty sensitive to the channel map you feed it. So if there are any mistakes there it may struggle to sort correctly. So my last big piece of advice is to double check your channel map to make sure that is okay. Sometimes people forget that their data matrix (n x m) may be organized with channels not in a (1:n) order but something random like (1,3,4,2). So making sure the probe->headstage wiring (that you make in Matlab) is done correctly can make a world of difference.

Zach

[neurojak]

Hi, I also recently reinstalled (using current instructions for dev branch) Phy and get the same "QMimeDatabase: Error loading internal MIME data" error described above, and had a permissions error when trying to use the yml file approach from the attachement above. Phy still loads, but I can't see the amplitude view. Any help would be greatly appreciated. I'm using Win10 Enterprise and have used previous a Phy2 version on the same computer without problems.

[zm711]

@neurojak What's the permission error?

[neurojak]

Hi Zach,

Sorry, the "permissions error" was relative to trying the original fix of using the dev instructions: (base) C:\Users\labadmin>git clone git@github.com:cortex-lab/phy.git Cloning into 'phy'... Warning: Permanently added the ECDSA host key for IP address '140.82.113.3' to the list of known hosts. git@github.com: Permission denied (publickey). fatal: Could not read from remote repository.

The YML error is: (base) C:\Users\labadmin.conda\envs>conda env create -f environment.yml

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

Traceback (most recent call last):
  File "C:\Miniconda3\lib\site-packages\conda\exceptions.py", line 1125, in __call__
    return func(*args, **kwargs)
  File "C:\Miniconda3\lib\site-packages\conda_env\cli\main.py", line 80, in do_call
    exit_code = getattr(module, func_name)(args, parser)
  File "C:\Miniconda3\lib\site-packages\conda\notices\core.py", line 72, in wrapper
    return_value = func(*args, **kwargs)
  File "C:\Miniconda3\lib\site-packages\conda_env\cli\main_create.py", line 103, in execute
    spec = specs.detect(name=name, filename=get_filename(args.file), directory=os.getcwd())
  File "C:\Miniconda3\lib\site-packages\conda_env\specs\__init__.py", line 47, in detect
    if spec.can_handle():
  File "C:\Miniconda3\lib\site-packages\conda_env\specs\yaml_file.py", line 19, in can_handle
    self._environment = env.from_file(self.filename)
  File "C:\Miniconda3\lib\site-packages\conda_env\env.py", line 169, in from_file
    return from_yaml(yamlstr, filename=filename)
  File "C:\Miniconda3\lib\site-packages\conda_env\env.py", line 143, in from_yaml
    data = yaml_safe_load(yamlstr)
  File "C:\Miniconda3\lib\site-packages\conda\common\serialize.py", line 60, in yaml_safe_load
    return yaml.safe_load(string, version="1.2")
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\main.py", line 980, in safe_load
    return load(stream, SafeLoader, version)
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\main.py", line 935, in load
    return loader._constructor.get_single_data()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\constructor.py", line 109, in get_single_data
    node = self.composer.get_single_node()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\composer.py", line 78, in get_single_node
    document = self.compose_document()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\composer.py", line 104, in compose_document
    self.parser.get_event()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\parser.py", line 163, in get_event
    self.current_event = self.state()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\parser.py", line 239, in parse_document_end
    token = self.scanner.peek_token()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\scanner.py", line 182, in peek_token
    self.fetch_more_tokens()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\scanner.py", line 282, in fetch_more_tokens
    return self.fetch_value()
  File "C:\Miniconda3\lib\site-packages\ruamel_yaml\scanner.py", line 651, in fetch_value
    raise ScannerError(
ruamel_yaml.scanner.ScannerError: mapping values are not allowed here
  in "<unicode string>", line 139, column 44:
     ...  name="description" content="phy: interactive visualization and  ...
                                         ^ (line: 139)

$ C:\Miniconda3\Scripts\conda-env-script.py create -f environment.yml

environment variables: CIO_TEST= CONDA_AUTO_UPDATE_CONDA=false CONDA_DEFAULT_ENV=base CONDA_EXE=C:\Miniconda3\condabin..\Scripts\conda.exe CONDA_EXES="C:\Miniconda3\condabin..\Scripts\conda.exe" CONDA_PREFIX=C:\Miniconda3 CONDA_PROMPT_MODIFIER=(base) CONDA_PYTHON_EXE=C:\Miniconda3\python.exe CONDA_ROOT=C:\Miniconda3 CONDA_SHLVL=1 CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7 CURL_CA_BUNDLE= GENICAM_GENTL32_PATH=C:\Program Files\FLIR Systems\Spinnaker\cti\vs2015 GENICAM_GENTL64_PATH=C:\Program Files\FLIR Systems\Spinnaker\cti64\vs2015 HOMEPATH=\Users\labadmin NVTOOLSEXT_PATH=C:\Program Files\NVIDIA Corporation\NvToolsExt\ PATH=C:\Miniconda3;C:\Miniconda3\Library\mingw-w64\bin;C:\Miniconda3\Librar y\usr\bin;C:\Miniconda3\Library\bin;C:\Miniconda3\Scripts;C:\Miniconda 3\bin;C:\Miniconda3;C:\Miniconda3\Library\mingw-w64\bin;C:\Miniconda3\ Library\usr\bin;C:\Miniconda3\Library\bin;C:\Miniconda3\Scripts;C:\Min iconda3\bin;C:\Miniconda3\condabin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\libnvvp;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\libnvvp;C:\Program Files\AdoptOpenJDK\jdk8u202-b08- jre\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WIN DOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Pr ogram Files (x86)\National Instruments\Shared\LabVIEW CLI;C:\Program Files (x86)\IVI Foundation\VISA\WinNT\Bin;C:\Program Files\IVI Foundation\VISA\Win64\Bin;C:\Program Files (x86)\IVI Foundation\VISA\WinNT\Bin;C:\Program Files (x86)\IVI Foundation\IVI\bin;C:\Program Files\IVI Foundation\IVI\bin;C:\Program Files (x86)\Pololu\Tic\bin;C:\Program Files (x86)\Microsoft SQL Server\80\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn;C:\Program Files\MATLAB\R2022b\runtime\win64;C:\Program Files\MATLAB\R2022b\bin;C:\Program Files\MATLAB\R2019b\bin;C:\Program Files\Microsoft SQL Server\110\Tools\Binn;C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.0;C:\Program Files\Microsoft SQL Server\120\Tools\Binn;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\WINDOWS\system32; C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerSh ell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files\FLIR Systems\Spinnaker\bin64\vs2015;C:\Program Files\FLIR Systems\Spinnaker\bin\vs2015;C:\Program Files\NVIDIA Corporation\Nsight Compute 2022.2.0;C:\Users\labadmin\AppData\Roaming\ Python\Python38\Scripts;C:\Users\labadmin\AppData\Local\Microsoft\Wind owsApps;. PSMODULEPATH=C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\Windows PowerShell\v1.0\Modules;C:\Program Files\Intel\Wired Networking\ REQUESTS_CA_BUNDLE= SSL_CERT_FILE= VXIPNPPATH=C:\Program Files (x86)\IVI Foundation\VISA\

 active environment : base
active env location : C:\Miniconda3
        shell level : 1
   user config file : C:\Users\labadmin\.condarc

populated config files : C:\Users\labadmin.condarc conda version : 4.14.0 conda-build version : not installed python version : 3.8.13.final.0 virtual packages : cuda=12.0=0 win=0=0 __archspec=1=x86_64 base environment : C:\Miniconda3 (read only) conda av data dir : C:\Miniconda3\etc\conda conda av metadata url : None channel URLs : https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch package cache : C:\Miniconda3\pkgs C:\Users\labadmin.conda\pkgs C:\Users\labadmin\AppData\Local\conda\conda\pkgs envs directories : C:\Users\labadmin.conda\envs C:\Miniconda3\envs C:\Users\labadmin\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/4.14.0 requests/2.27.1 CPython/3.8.13 Windows/10 Windows/10.0.19045 administrator : False netrc file : None offline mode : False

An unexpected error has occurred. Conda has prepared the above report.

If submitted, this report will be used by core maintainers to improve future releases of conda. Would you like conda to send this report to the core maintainers?

[y/N]:

[zm711]

What are the contents of the yaml file? The file should say this

name: phy2
channels:
  - conda-forge
  - defaults
dependencies:
  - python
  - pip
  - git
  - numpy
  - matplotlib
  - scipy
  - h5py
  - pyqt
  - pyopengl
  - pyqtwebengine
  - pytest
  - qtconsole
  - requests
  - responses
  - traitlets
  - dask
  - cython
  - pillow
  - scikit-learn
  - joblib
  - pip:
    - git+https://github.com/cortex-lab/phy.git

Alternatively you can right click this file and download and use instead environment.yml

The dev instructions/file are out of date. Could you try with the new file instead and let me know how that goes.

[neurojak]

Hmmm, OK there is something up with the yml file - I haven't used them much before, never written one. When I download the file you just linked and open in a text editor, it is filled with markup language and quite long, starts with:

<!DOCTYPE html>

[neurojak]

<!DOCTYPE html>

[neurojak]

OK, Github is interpreting the markup when I paste it...let me try again:

[zm711]

You're right. Instead can you just copy the file into a text editor (vscode is nice but notepad can work fine too). In my above comment I gave you the exact install you need. Or you can clone the repo and use the file (but since cloning failed for you). Just copy this:

name: phy2
channels:
  - conda-forge
  - defaults
dependencies:
  - python
  - pip
  - git
  - numpy
  - matplotlib
  - scipy
  - h5py
  - pyqt
  - pyopengl
  - pyqtwebengine
  - pytest
  - qtconsole
  - requests
  - responses
  - traitlets
  - dask
  - cython
  - pillow
  - scikit-learn
  - joblib
  - pip:
    - git+https://github.com/cortex-lab/phy.git

[neurojak]

Thanks! That yml install worked but took a while. Now the "QMimeDatabase: Error loading internal MIME data" error is resolved. I'm still not seeing amplitude view for some reason, but I'll raise that on another issue if I can't figure out why.

[neurojak]

FYI...amplitude view problem was related to issue #1175, fixed now, thanks again for the help

[Hobart10]

Hi Zach, @zm711 Sorry that I have same issue of no waveforms showing up with Kipkeller's first issue (after phy template-gui params.py in kilosort out dir), just seems no error in loading internal MIME data, but same steps still not working: I've tried 1) re-create env and install development version of phy; 2) create env from the environment.yml; 3) tested with neuropixel example dataset and our own dataset; 4) tested on two PC, one Mac M1 (all with anaconda prompt). All end up with these issue, phy gui appears but actually nothing is showing. Could you please review my procedure? Any insights would be highly appreciated!

Error in conda terminal: [W] model:603 Unreferenced clusters found in templates (generally not a problem) [W] model:625 Unreferenced clusters found in spike_clusters (generally not a problem) [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed.

[zm711]

@Hobart10

No worries. We can try to figure this out. There isn't actually an error in what you shared. The data often needs to extract from raw data on the fly. So I think what would be beneficial is if you could run:

phy template-gui params.py --debug

And copy everything that shows up there into the prompt, so I can see how your model is being generated. It would also be beneficial if you did

conda list

So I can see what is in your environment to make sure you installed your environment correctly.

Finally,

it will hep me read things if you use `. That is the backtick typically near the upper left of the keyboard. If you use 3 backticks at the beginning of your text box and at the end for the debug and conda list, then it will format to make it easier to read for me

an example

[kipkeller]

@Hobart10 Just to be sure it isn't something really simple. You do have to choose a cluster (in clusterview) before anything will show up in waveformview.... kip

[Hobart10]

@Hobart10

No worries. We can try to figure this out. There isn't actually an error in what you shared. The data often needs to extract from raw data on the fly. So I think what would be beneficial is if you could run:

phy template-gui params.py --debug

And copy everything that shows up there into the prompt, so I can see how your model is being generated. It would also be beneficial if you did

conda list

So I can see what is in your environment to make sure you installed your environment correctly.

Finally,

it will hep me read things if you use `. That is the backtick typically near the upper left of the keyboard. If you use 3 backticks at the beginning of your text box and at the end for the debug and conda list, then it will format to make it easier to read for me

an example

Hi Zach, Thank you for your external support! Sorry for the late response, the waveforms appeared with choosing one cluster as Kip suggested. Thanks to Kip!

FYI, Debug output:

17:26:08.564 [D] __init__:68          Start capturing exceptions.
17:26:08.620 [W] model:603            Unreferenced clusters found in templates (generally not a problem)
17:26:08.624 [D] model:619            Loading spike clusters.
17:26:08.636 [W] model:625            Unreferenced clusters found in spike_clusters (generally not a problem)
17:26:08.647 [D] model:569            No channel shank file found.
17:26:08.648 [D] model:692            Loading templates.
17:26:08.678 [D] model:724            Templates are dense.
17:26:08.679 [W] model:667            Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed.
17:26:08.679 [D] model:730            Loading the whitening matrix.
17:26:08.683 [D] model:737            Loading the inverse of the whitening matrix.
17:26:08.696 [D] model:766            Loading features.
17:26:08.697 [D] model:803            Loading template features.
17:26:08.699 [D] model:504            Load `cluster_Amplitude.tsv`.
17:26:08.702 [D] model:504            Load `cluster_ContamPct.tsv`.
17:26:08.704 [D] model:504            Load `cluster_group.tsv`.
17:26:08.707 [D] model:504            Load `cluster_KSLabel.tsv`.
17:26:08.974 [D] context:100          Initialize joblib cache dir at `\\10.50.4.1\Personal\Yukai Xu\EphysData\441EphysData\Wireless\230823\kiloOut\.phy`.
17:26:08.976 [D] context:101          Reducing the size of the cache if needed.
17:26:09.131 [D] base:102             Add filter `high_pass`.
17:26:09.133 [D] config:31            Load config file `C:\Users\Lenovo\.phy\phy_config.py`.
17:26:09.135 [D] plugin:145           Loading 0 plugins.
17:26:09.136 [D] context:126          Load memcache for `phy.apps.base._get_mean_waveforms`.
17:26:09.139 [D] context:126          Load memcache for `phy.apps.base._get_mean_waveforms`.
17:26:09.141 [D] context:126          Load memcache for `phy.apps.base._get_template_waveforms`.
17:26:09.146 [D] context:126          Load memcache for `phy.apps.base.get_mean_spike_template_amplitudes`.
17:26:09.149 [D] context:126          Load memcache for `phy.apps.base.get_template_counts`.
17:26:09.152 [D] context:126          Load memcache for `phy.apps.base.get_template_for_cluster`.
17:26:09.155 [D] context:126          Load memcache for `phy.apps.template.gui.get_template_amplitude`.
17:26:09.158 [D] context:126          Load memcache for `phy.apps.base.get_cluster_amplitude`.
17:26:09.161 [D] context:126          Load memcache for `phy.apps.base.get_mean_firing_rate`.
17:26:09.165 [D] context:126          Load memcache for `phy.apps.base.get_best_channel`.
17:26:09.169 [D] context:126          Load memcache for `phy.apps.template.gui.get_best_channels`.
17:26:09.172 [D] context:126          Load memcache for `phy.apps.base.get_channel_shank`.
17:26:09.175 [D] context:126          Load memcache for `phy.apps.base.get_probe_depth`.
17:26:09.178 [D] context:126          Load memcache for `phy.apps.base.peak_channel_similarity`.
17:26:09.183 [D] context:209          The file `\\10.50.4.1\Personal\Yukai Xu\EphysData\441EphysData\Wireless\230823\kiloOut\.phy\new_cluster_id.pkl` doesn't exist.
17:26:09.207 [D] context:185          Save data to `\\10.50.4.1\Personal\Yukai Xu\EphysData\441EphysData\Wireless\230823\kiloOut\.phy\spikes_per_cluster.pkl`.
17:26:09.252 [D] gui:463              Creating GUI.
17:26:09.265 [D] state:46             Load C:\Users\Lenovo\.phy\TemplateGUI\state.json for GUIState.
17:26:09.267 [D] state:46             Load \\10.50.4.1\Personal\Yukai Xu\EphysData\441EphysData\Wireless\230823\kiloOut\.phy\state.json for GUIState.
17:26:09.589 [D] gui:718              Add view ClusterView to GUI.
17:26:09.598 [D] gui:718              Add view SimilarityView to GUI.
17:26:09.628 [D] gui:718              Add view WaveformView to GUI.
17:26:09.869 [D] base:337             Set state for WaveformView.
17:26:09.879 [D] gui:718              Add view CorrelogramView to GUI.
17:26:09.883 [D] base:337             Set state for CorrelogramView.
17:26:09.904 [D] gui:718              Add view ISIView to GUI.
17:26:09.907 [D] base:337             Set state for ISIView.
17:26:09.930 [D] gui:718              Add view AmplitudeView to GUI.
17:26:09.933 [D] base:337             Set state for AmplitudeView.
17:26:09.952 [D] gui:718              Add view FiringRateView to GUI.
17:26:09.955 [D] base:337             Set state for FiringRateView.
17:26:09.967 [D] gui:718              Add view ProbeView to GUI.
17:26:09.970 [D] base:337             Set state for ProbeView.
17:26:10.077 [D] gui:519              Load the geometry state.
17:26:10.576 [D] supervisor:817       Clusters selected: [2] (3)
17:26:12.761 [D] supervisor:817       Clusters selected: [0] (1)
17:26:14.176 [D] supervisor:817       Clusters selected: [1] (2)
17:26:14.983 [D] supervisor:817       Clusters selected: [2] (3)
17:26:15.700 [D] supervisor:817       Clusters selected: [4] (5)
17:26:16.452 [D] supervisor:817       Clusters selected: [6] (7)
17:26:17.219 [D] supervisor:817       Clusters selected: [7] (8)

conda list:

# packages in environment at C:\Users\Lenovo\.conda\envs\phy2:
#
# Name                    Version                   Build  Channel
asttokens                 2.4.0              pyhd8ed1ab_0    conda-forge
aws-c-auth                0.7.3                hd125877_3    conda-forge
aws-c-cal                 0.6.2                hfb91821_0    conda-forge
aws-c-common              0.9.0                hcfcfb64_0    conda-forge
aws-c-compression         0.2.17               h04c9df6_2    conda-forge
aws-c-event-stream        0.3.2                h495bb32_0    conda-forge
aws-c-http                0.7.12               h0890e15_1    conda-forge
aws-c-io                  0.13.32              h83b3346_3    conda-forge
aws-c-mqtt                0.9.5                h0fd1aac_1    conda-forge
aws-c-s3                  0.3.17               h9f49523_0    conda-forge
aws-c-sdkutils            0.1.12               h04c9df6_1    conda-forge
aws-checksums             0.1.17               h04c9df6_1    conda-forge
aws-crt-cpp               0.23.1               hfe9bf68_1    conda-forge
aws-sdk-cpp               1.11.156             h77892aa_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
bokeh                     3.2.2              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hcfcfb64_0    conda-forge
brotli-bin                1.1.0                hcfcfb64_0    conda-forge
brotli-python             1.1.0           py311h12c1d0e_0    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
c-ares                    1.19.1               hcfcfb64_0    conda-forge
ca-certificates           2023.7.22            h56e8100_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
cloudpickle               2.2.1              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
colorcet                  3.0.1                    pypi_0    pypi
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.0           py311h005e61a_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cython                    3.0.2           py311h12c1d0e_0    conda-forge
cytoolz                   0.12.2          py311ha68e1ae_0    conda-forge
dask                      2023.9.1           pyhd8ed1ab_0    conda-forge
dask-core                 2023.9.1           pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311h12c1d0e_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
distributed               2023.9.1           pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
fonttools                 4.42.1          py311ha68e1ae_0    conda-forge
freetype                  2.12.1               hdaf720e_2    conda-forge
fsspec                    2023.9.0           pyh1a96a4e_0    conda-forge
gettext                   0.21.1               h5728263_0    conda-forge
ghp-import                2.1.0                    pypi_0    pypi
git                       2.42.0               h57928b3_0    conda-forge
glib                      2.78.0               h12be248_0    conda-forge
glib-tools                2.78.0               h12be248_0    conda-forge
gst-plugins-base          1.22.5               h001b923_1    conda-forge
gstreamer                 1.22.5               hb4038d2_1    conda-forge
h5py                      3.9.0           nompi_py311h0d04526_102    conda-forge
hdf5                      1.14.2          nompi_h73e8ff5_100    conda-forge
icu                       73.2                 h63175ca_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
intel-openmp              2023.2.0         h57928b3_49496    conda-forge
ipykernel                 6.25.2             pyh60829e3_0    conda-forge
ipython                   8.15.0             pyh5737063_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
jedi                      0.19.0             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.3.1              pyhd8ed1ab_0    conda-forge
jupyter_core              5.3.1           py311h1ea47a8_0    conda-forge
kiwisolver                1.4.5           py311h005e61a_0    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
lcms2                     2.15                 he9d350c_2    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libabseil                 20230802.0      cxx17_h63175ca_3    conda-forge
libaec                    1.0.6                h63175ca_1    conda-forge
libarrow                  13.0.0           h1e3473c_4_cpu    conda-forge
libblas                   3.9.0              18_win64_mkl    conda-forge
libbrotlicommon           1.1.0                hcfcfb64_0    conda-forge
libbrotlidec              1.1.0                hcfcfb64_0    conda-forge
libbrotlienc              1.1.0                hcfcfb64_0    conda-forge
libcblas                  3.9.0              18_win64_mkl    conda-forge
libclang                  15.0.7          default_h77d9078_3    conda-forge
libclang13                15.0.7          default_h77d9078_3    conda-forge
libcrc32c                 1.1.2                h0e60522_0    conda-forge
libcurl                   8.3.0                hd5e4a3a_0    conda-forge
libdeflate                1.18                 hcfcfb64_0    conda-forge
libevent                  2.1.12               h3671451_1    conda-forge
libexpat                  2.5.0                h63175ca_1    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libglib                   2.78.0               he8f3873_0    conda-forge
libgoogle-cloud           2.12.0               h0a0a397_2    conda-forge
libgrpc                   1.57.0               h550f6bd_1    conda-forge
libhwloc                  2.9.2           default_haede6df_1009    conda-forge
libiconv                  1.17                 h8ffe710_0    conda-forge
libjpeg-turbo             2.1.5.1              hcfcfb64_1    conda-forge
liblapack                 3.9.0              18_win64_mkl    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libpng                    1.6.39               h19919ed_0    conda-forge
libprotobuf               4.23.4               hb8276f3_6    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsqlite                 3.43.0               hcfcfb64_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libthrift                 0.19.0               h06f6336_0    conda-forge
libtiff                   4.6.0                h6c8260b_0    conda-forge
libutf8proc               2.8.0                h82a8f57_0    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp                   1.3.1                hcfcfb64_1    conda-forge
libwebp-base              1.3.1                hcfcfb64_0    conda-forge
libxcb                    1.15                 hcd874cb_0    conda-forge
libxml2                   2.11.5               hc3477c8_1    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lz4                       4.3.2           py311haddf500_0    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markdown                  3.4.4                    pypi_0    pypi
markupsafe                2.1.3           py311ha68e1ae_0    conda-forge
matplotlib                3.7.2           py311h1ea47a8_0    conda-forge
matplotlib-base           3.7.2           py311h6e989c2_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mergedeep                 1.3.4                    pypi_0    pypi
mkdocs                    1.5.2                    pypi_0    pypi
mkl                       2022.1.0           h6a75c08_874    conda-forge
msgpack-python            1.0.5           py311h005e61a_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
mtscomp                   1.0.2                    pypi_0    pypi
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311h0b4df5a_0    conda-forge
openjpeg                  2.5.0                h3d672ee_3    conda-forge
openssl                   3.1.2                hcfcfb64_0    conda-forge
orc                       1.9.0                h8dbeef6_2    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pandas                    2.1.0           py311hf63dbb6_0    conda-forge
param                     1.13.0                   pypi_0    pypi
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.4.0              pyhd8ed1ab_1    conda-forge
pathspec                  0.11.2                   pypi_0    pypi
pcre2                     10.40                h17e33f8_0    conda-forge
phy                       2.0b5                    pypi_0    pypi
phylib                    2.4.3                    pypi_0    pypi
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.0.0          py311hd926f49_1    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
platformdirs              3.10.0             pyhd8ed1ab_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
ply                       3.11                       py_1    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311ha68e1ae_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   13.0.0          py311h6a6099b_4_cpu    conda-forge
pyct                      0.5.0                    pypi_0    pypi
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pyopengl                  3.1.6              pyhd8ed1ab_1    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyqt                      5.15.9          py311h125bc19_4    conda-forge
pyqt5-sip                 12.12.2         py311h12c1d0e_4    conda-forge
pyqtwebengine             5.15.9          py311h5a77453_4    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
pytest                    7.4.2              pyhd8ed1ab_0    conda-forge
python                    3.11.5          h2628c8c_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    3_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pywin32                   304             py311h12c1d0e_2    conda-forge
pyyaml                    6.0.1           py311ha68e1ae_0    conda-forge
pyyaml-env-tag            0.1                      pypi_0    pypi
pyzmq                     25.1.1          py311h7b3f143_0    conda-forge
qt-main                   5.15.8              he5a7383_16    conda-forge
qt-webengine              5.15.8               h5b1ea0b_0    conda-forge
qtconsole                 5.4.4              pyhd8ed1ab_0    conda-forge
qtconsole-base            5.4.4              pyha770c72_0    conda-forge
qtpy                      2.4.0              pyhd8ed1ab_0    conda-forge
re2                       2023.03.02           hd4eee63_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
responses                 0.23.2             pyhd8ed1ab_1    conda-forge
scikit-learn              1.3.0           py311h142b183_0    conda-forge
scipy                     1.11.2          py311h37ff6ca_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
sip                       6.7.11          py311h12c1d0e_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               hfb803bf_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tbb                       2021.10.0            h91493d7_0    conda-forge
tblib                     2.0.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311ha68e1ae_0    conda-forge
tqdm                      4.66.1                   pypi_0    pypi
traitlets                 5.10.0             pyhd8ed1ab_0    conda-forge
types-pyyaml              6.0.12.11          pyhd8ed1ab_0    conda-forge
typing-extensions         4.7.1                hd8ed1ab_0    conda-forge
typing_extensions         4.7.1              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
urllib3                   2.0.4              pyhd8ed1ab_0    conda-forge
vc                        14.3                h64f974e_17    conda-forge
vc14_runtime              14.36.32532         hdcecf7f_17    conda-forge
vs2015_runtime            14.36.32532         h05e6639_17    conda-forge
watchdog                  3.0.0                    pypi_0    pypi
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xorg-libxau               1.0.11               hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xyzservices               2023.7.0           pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zeromq                    4.3.4                h0e60522_1    conda-forge
zict                      3.0.0              pyhd8ed1ab_0    conda-forge
zipp                      3.16.2             pyhd8ed1ab_0    conda-forge
zstd                      1.5.5                h12be248_0    conda-forge