kingsEffy
commented
4 months ago The last AssertionError is cuz I generated feature npy by myself inside kilosort3, and apparentlly it was somehow invoking conflicts ...
Closed kingsEffy closed 1 month ago
kingsEffy
commented
4 months ago The last AssertionError is cuz I generated feature npy by myself inside kilosort3, and apparentlly it was somehow invoking conflicts ...
kingsEffy
commented
4 months ago besides I already tried installing/uninstalling phy many times on 3 different computer ... it is getting frustrated since im having some opto-tagging recordings that some small amplitude artifacts (after removal) still mixed up with real spikes ...
zm711
commented
4 months ago Howdy @kingsEffy.
I'm a bit worried that the install instructions are a bit out of date. You have MIME error in the screen shot you shared which actually means you have a broken install of Phy2. What you should do is remove your current install of Phy2. Then download the environment.yaml from the repo. Then edit the yaml to have python<3.12 (there is a bug with Phy and python 3.12 currently). Then create an environment with a file.
conda env create -f environment.ymlHowdy @kingsEffy.
I'm a bit worried that the install instructions are a bit out of date. You have MIME error in the screen shot you shared which actually means you have a broken install of Phy2. What you should do is remove your current install of Phy2. Then download the
environment.yamlfrom the repo. Then edit the yaml to have python<3.12 (there is a bug with Phy and python 3.12 currently). Then create an environment with a file.conda env create -f environment.yml
Hey @zm711 I did excatly as you told me, the errors remain:
I tried on two desktop.. none worked...
There must be something i did wrong, but how? and why?
zm711
commented
4 months ago Hey @kingsEffy, you cut off the top of the stack trace. Could I see all of the trace?
I'm trying to figure out if Phy is broken or if maybe the plugin no longer works.
kingsEffy
commented
4 months ago This is from another computer but outcome is the same (I just left another desktop):
(phy2) C:..........................(my data directory)>phy template-gui params.py --debug
16:50:25.827 [D] init:68 Start capturing exceptions.
16:50:25.851 [D] model:619 Loading spike clusters.
16:50:25.880 [D] model:569 No channel shank file found.
16:50:25.882 [D] model:692 Loading templates.
16:50:25.887 [D] model:724 Templates are dense.
16:50:25.890 [D] model:671 Loading spikes subset waveforms to avoid fetching waveforms from raw data.
16:50:25.894 [D] model:730 Loading the whitening matrix.
16:50:25.895 [D] model:737 Loading the inverse of the whitening matrix.
16:50:25.900 [D] model:766 Loading features.
16:50:25.900 [D] model:803 Loading template features.
16:50:25.903 [D] model:504 Load cluster_Amplitude.tsv.
16:50:25.904 [D] model:504 Load cluster_ContamPct.tsv.
16:50:25.906 [D] model:504 Load cluster_group.tsv.
16:50:25.908 [D] model:504 Load cluster_KSLabel.tsv.
16:50:26.335 [D] context:100 Initialize joblib cache dir at .(my data directory)\kilosort3\.phy.
16:50:26.336 [D] context:101 Reducing the size of the cache if needed.
16:50:26.344 [D] base:102 Add filter high_pass.
16:50:26.344 [D] config:31 Load config file C:\Users\effyw\.phy\phy_config.py.
16:50:26.360 [D] plugin:74 Scanning C:\Users\effyw\.phy\plugins.
16:50:26.361 [D] plugin:78 Found plugin module ControllerSettings.py.
16:50:26.377 [D] plugin:36 Register plugin ControllerSettings.ControllerSettings.
16:50:26.378 [D] plugin:78 Found plugin module ControllerSettingsKwik.py.
16:50:26.390 [D] plugin:36 Register plugin ControllerSettingsKwik.ControllerSettingsKwik.
16:50:26.391 [D] plugin:78 Found plugin module CustomActionPlugin.py.
16:50:26.394 [D] plugin:36 Register plugin CustomActionPlugin.CustomActionPlugin.
16:50:26.395 [D] plugin:78 Found plugin module export2neurosuite.py.
16:50:32.849 [D] plugin:36 Register plugin export2neurosuite.Export2neurosuite.
16:50:32.850 [D] plugin:78 Found plugin module n_spikes_views.py.
16:50:32.858 [D] plugin:36 Register plugin n_spikes_views.n_spikes_views.
16:50:32.859 [D] plugin:78 Found plugin module raw_data_filter.py.
16:50:32.882 [D] plugin:36 Register plugin raw_data_filter.RawDataFilterPlugin.
16:50:32.883 [D] plugin:78 Found plugin module recluster.py.
16:50:32.998 [D] plugin:36 Register plugin recluster.Recluster.
16:50:32.999 [D] plugin:78 Found plugin module SplitShortISI.py.
16:50:33.027 [D] plugin:36 Register plugin SplitShortISI.SplitShortISI.
16:50:33.027 [D] plugin:78 Found plugin module SplitShortISI_v2.py.
16:50:33.050 [D] plugin:36 Register plugin SplitShortISI_v2.SplitShortISI.
16:50:33.051 [D] plugin:78 Found plugin module tempdir.py.
16:50:33.060 [D] plugin:145 Loading 4 plugins.
16:50:33.061 [D] plugin:156 Attached plugin Recluster.
16:50:33.063 [D] base:102 Add filter high_pass.
16:50:33.063 [D] plugin:156 Attached plugin RawDataFilterPlugin.
16:50:33.064 [D] plugin:156 Attached plugin CustomActionPlugin.
16:50:33.068 [D] plugin:156 Attached plugin SplitShortISI.
16:50:33.069 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms.
16:50:33.083 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms.
16:50:33.085 [D] context:126 Load memcache for phy.apps.base._get_template_waveforms.
16:50:33.087 [D] context:126 Load memcache for phy.apps.base.get_mean_spike_template_amplitudes.
16:50:33.090 [D] context:126 Load memcache for phy.apps.base.get_template_counts.
16:50:33.106 [D] context:126 Load memcache for phy.apps.base.get_template_for_cluster.
16:50:33.121 [D] context:126 Load memcache for phy.apps.template.gui.get_template_amplitude.
16:50:33.136 [D] context:126 Load memcache for phy.apps.base.get_cluster_amplitude.
16:50:33.139 [D] context:126 Load memcache for phy.apps.base.get_mean_firing_rate.
16:50:33.141 [D] context:126 Load memcache for phy.apps.base.get_best_channel.
16:50:33.154 [D] context:126 Load memcache for phy.apps.template.gui.get_best_channels.
16:50:33.165 [D] context:126 Load memcache for phy.apps.base.get_channel_shank.
16:50:33.167 [D] context:126 Load memcache for phy.apps.base.get_probe_depth.
16:50:33.170 [D] context:126 Load memcache for phy.apps.base.peak_channel_similarity.
16:50:33.175 [D] context:209 The file .(my data directory)\kilosort3\.phy\new_cluster_id.pkl doesn't exist.
16:50:33.211 [D] context:185 Save data to .(my data directory)\.phy\spikes_per_cluster.pkl.
16:50:33.224 [D] gui:463 Creating GUI.
16:50:33.249 [D] state:46 Load C:\Users\effyw.phy\TemplateGUI\state.json for GUIState.
16:50:33.264 [D] state:46 Load .(my data directory)\kilosort3.phy\state.json for GUIState.
16:50:33.666 [D] gui:718 Add view ClusterView to GUI.
16:50:33.678 [D] gui:718 Add view SimilarityView to GUI.
16:50:33.722 [D] gui:718 Add view WaveformView to GUI.
16:50:33.963 [D] base:337 Set state for WaveformView.
16:50:33.979 [D] gui:718 Add view CorrelogramView to GUI.
16:50:33.981 [D] base:337 Set state for CorrelogramView.
16:50:34.005 [D] gui:718 Add view ISIView to GUI.
16:50:34.008 [D] base:337 Set state for ISIView.
16:50:34.025 [D] gui:718 Add view FeatureView to GUI.
16:50:34.028 [D] base:337 Set state for FeatureView.
16:50:34.060 [D] gui:718 Add view AmplitudeView to GUI.
16:50:34.063 [D] base:337 Set state for AmplitudeView.
16:50:34.091 [D] gui:718 Add view FiringRateView to GUI.
16:50:34.094 [D] base:337 Set state for FiringRateView.
16:50:34.163 [D] gui:718 Add view TraceView to GUI.
16:50:34.166 [D] base:337 Set state for TraceView.
16:50:34.166 [D] base:494 Set color scheme to random.
16:50:34.808 [D] base:1626 Adding default color schemes to TraceView.
16:50:35.191 [D] gui:718 Add view ProbeView to GUI.
16:50:35.195 [D] base:337 Set state for ProbeView.
16:50:35.227 [D] gui:718 Add view ClusterScatterView to GUI.
16:50:35.230 [D] base:337 Set state for ClusterScatterView.
16:50:35.230 [D] base:494 Set color scheme to random.
16:50:35.232 [D] base:1626 Adding default color schemes to ClusterScatterView.
16:50:35.462 [D] gui:519 Load the geometry state.
16:50:36.337 [D] supervisor:817 Clusters selected: [14] (27)
16:50:53.739 [D] supervisor:839 Similar clusters selected: [12] (9)
16:50:57.408 [D] actions:67 Prompting arguments for MahalanobisDist
16:50:58.722 [W] recluster:218 Removing outliers by Mahalanobis distance
16:50:58.757 [D] model:766 Loading features.
16:50:58.758 [W] actions:213 Error when executing action MahalanobisDist.
16:50:58.761 [D] actions:214 Traceback (most recent call last):
File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phy\gui\actions.py", line 211, in wrapped
return callback(*args)
^^^^^^^^^^^^^^^
File "C:\Users\effyw.phy\plugins\recluster.py", line 252, in MahalanobisDist
data3 = data.data[spike_ids]
^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'data'
16:51:18.961 [D] actions:67 Prompting arguments for K_means_clustering 16:51:20.169 [W] recluster:195 Running K-means clustering 16:51:20.200 [D] model:766 Loading features. 16:51:20.201 [W] actions:213 Error when executing action K means clustering. 16:51:20.202 [D] actions:214 Traceback (most recent call last): File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phy\gui\actions.py", line 211, in wrapped return callback(*args) ^^^^^^^^^^^^^^^ File "C:\Users\effyw.phy\plugins\recluster.py", line 203, in K_means_clustering data3 = data.data[spike_ids] ^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'data'
16:51:36.265 [I] supervisor:702 Assigned 116483 spikes.
16:51:36.266 [D] context:185 Save data to my data directory\.phy\new_cluster_id.json.
16:51:36.316 [D] supervisor:817 Clusters selected: [28, 30] (27)
16:51:42.320 [D] actions:67 Prompting arguments for MahalanobisDist
16:51:43.601 [W] recluster:218 Removing outliers by Mahalanobis distance
16:51:43.631 [D] model:766 Loading features.
16:51:43.632 [W] actions:213 Error when executing action MahalanobisDist.
16:51:43.633 [D] actions:214 Traceback (most recent call last):
File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phy\gui\actions.py", line 211, in wrapped
return callback(*args)
^^^^^^^^^^^^^^^
File "C:\Users\effyw.phy\plugins\recluster.py", line 252, in MahalanobisDist
data3 = data.data[spike_ids]
^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'data'
16:51:45.249 [D] actions:67 Prompting arguments for K_means_clustering 16:51:45.841 [W] recluster:195 Running K-means clustering 16:51:45.870 [D] model:766 Loading features. 16:51:45.871 [W] actions:213 Error when executing action K means clustering. 16:51:45.872 [D] actions:214 Traceback (most recent call last): File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phy\gui\actions.py", line 211, in wrapped return callback(*args) ^^^^^^^^^^^^^^^ File "C:\Users\effyw.phy\plugins\recluster.py", line 203, in K_means_clustering data3 = data.data[spike_ids] ^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'data'
16:51:50.044 [I] recluster:71 Running KlustaKwik on 206 spikes. 16:51:50.046 [D] model:766 Loading features. 16:51:50.049 [W] actions:213 Error when executing action Recluster Local PCAs. 16:51:50.050 [D] actions:214 Traceback (most recent call last): File "C:\ProgramData\anaconda3\envs\phy2\Lib\site-packages\phy\gui\actions.py", line 211, in wrapped return callback(*args) ^^^^^^^^^^^^^^^ File "C:\Users\effyw.phy\plugins\recluster.py", line 73, in Recluster_Local_PCAs data3 = controller.model._load_features().data[spike_ids] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'data'
16:52:15.076 [D] state:171 Update GUI state for ClusterView
16:52:15.093 [D] state:171 Update GUI state for WaveformView
16:52:15.094 [D] state:171 Update GUI state for CorrelogramView
16:52:15.096 [D] state:171 Update GUI state for ISIView
16:52:15.098 [D] state:171 Update GUI state for FeatureView
16:52:15.099 [D] state:171 Update GUI state for AmplitudeView
16:52:15.101 [D] state:171 Update GUI state for FiringRateView
16:52:15.102 [D] state:171 Update GUI state for TraceView
16:52:15.104 [D] state:171 Update GUI state for ProbeView
16:52:15.105 [D] state:171 Update GUI state for ClusterScatterView
16:52:17.489 [D] context:138 Save memcache for phy.apps.base._get_mean_waveforms.
16:52:17.491 [D] context:138 Save memcache for phy.apps.base._get_template_waveforms.
16:52:17.494 [D] context:138 Save memcache for phy.apps.base.get_mean_spike_template_amplitudes.
16:52:17.495 [D] context:138 Save memcache for phy.apps.base.get_template_counts.
16:52:17.496 [D] context:138 Save memcache for phy.apps.base.get_template_for_cluster.
16:52:17.497 [D] context:138 Save memcache for phy.apps.template.gui.get_template_amplitude.
16:52:17.498 [D] context:138 Save memcache for phy.apps.base.get_cluster_amplitude.
16:52:17.498 [D] context:138 Save memcache for phy.apps.base.get_mean_firing_rate.
16:52:17.499 [D] context:138 Save memcache for phy.apps.base.get_best_channel.
16:52:17.500 [D] context:138 Save memcache for phy.apps.template.gui.get_best_channels.
16:52:17.501 [D] context:138 Save memcache for phy.apps.base.get_channel_shank.
16:52:17.502 [D] context:138 Save memcache for phy.apps.base.get_probe_depth.
16:52:17.502 [D] context:138 Save memcache for phy.apps.base.peak_channel_similarity.
16:52:17.503 [D] gui:600 Save the geometry state.
16:52:17.504 [D] state:218 Save global GUI state to C:\Users\effyw\.phy\TemplateGUI\state.json.
16:52:17.506 [D] state:232 Save local GUI state to .(my data directory)\kilosort3\.phy\state.json.
16:52:17.544 [D] model:263 Close memmap array sparse_clusters.data.
16:52:17.544 [D] model:263 Close memmap array sparse_templates.data.
16:52:17.546 [D] model:263 Close memmap array spike_waveforms.waveforms.
16:52:17.548 [D] init:87 Stop capturing exceptions.
kingsEffy
commented
4 months ago Hey @kingsEffy, you cut off the top of the stack trace. Could I see all of the trace?
I'm trying to figure out if Phy is broken or if maybe the plugin no longer works.
I just posted all the outputs in the termial, see above...
zm711
commented
4 months ago Based on my reading it looks like Phy is now installed correctly. But the plugins are not behaving correctly. Did you just find the plugins on line or is it from a labmate that you could ask to help? Phy is finding and loading the plugins, but while running the plugins the plugins themselves are failing. I'm not familiar with this particular groups plug-ins but I can read up on them unless you are in this lab and can ask.
zm711
commented
4 months ago After doing a quick read it seems like these plug-ins would only work on an older version of phy2. So you could try to play around with an older version, but it is probably best to just use phy2 without the plugins for now and see if works without them first.
kingsEffy
commented
4 months ago the plugins are from here: https://github.com/petersenpeter/phy2-plugins
let me try again using this version to do some curation first...
Thank you so much! At least I'm getting an idea of what's going on!
zm711
commented
4 months ago Just for reference I'm referring to this comment: https://github.com/petersenpeter/phy2-plugins/issues/3#issuecomment-921909062. Unfortunately the other person's solution to downgrade to an older phy2 is very difficult to do at this point since the requirements.txt has been updated. It will basically always break.
But happy to help. Just let me know if you have trouble using phy without the plugins!
kingsEffy
commented
4 months ago Many thanks to you. I think I’m gonna move on and utilize the plugins in phy git. I agree it’s complicated to try to use his one. You provovided me with very clear instructions. I have been wasting my whole week to fix this!
Get Outlook for iOShttps://aka.ms/o0ukef
From: Zach McKenzie @.> Sent: Friday, February 9, 2024 5:29:50 PM To: cortex-lab/phy @.> Cc: kingsEffy @.>; Comment @.> Subject: Re: [cortex-lab/phy] unable to do curation using phy2 (Issue #1255)
Just for reference I'm referring to this comment: petersenpeter/phy2-plugins#3 (comment)https://github.com/petersenpeter/phy2-plugins/issues/3#issuecomment-921909062. Unfortunately the other person's solution to downgrade to an older phy2 is very difficult to do at this point since the requirements.txt has been updated. It will basically always break.
But happy to help. Just let me know if you have trouble using phy without the plugins!
— Reply to this email directly, view it on GitHubhttps://github.com/cortex-lab/phy/issues/1255#issuecomment-1936796537, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AX75ER2PRJR7GPILHJILDR3YS3EQ5AVCNFSM6AAAAABDAIENIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZWG44TMNJTG4. You are receiving this because you commented.Message ID: @.***>
hi I have been struggleing to use phy2 to do psot-sorting curations inside Phy2. The installation came from here: https://github.com/cortex-lab/phy/ . Very striaghtforward and simple. However when i add plugin from Peterson https://github.com/petersenpeter/phy2-plugins I received errors:
Anyone who knows how to solve this?
Let me know!