search

cortex-lab / phy

phy: interactive visualization and manual spike sorting of large-scale ephys data
BSD 3-Clause "New" or "Revised" License
318 stars 157 forks source link

Cannot open TemplateFeatureView #1290

Open florian6973 opened 4 months ago

florian6973 commented 4 months ago

Hi!

I am opening an issue about the Template Feature View, because I try to export a WaveformExtractor from spikeinterface (0.100.8) to phy, but then I cannot open the Template Feature View (it does not appear). I saw there are a few issues opened about this issue but they only focus on KS3 and not through the spikeinterface export. I would be very grateful if you have any ideas of why it is happening and potentially how to solve it. Thanks :)

Log:

(phy2) C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy>phy template-gui params.py --debug 14:31:24.320 [D] init:68 Start capturing exceptions. 14:31:24.320 [W] model:1408 File None does not exist. 14:31:24.320 [D] model:619 Loading spike clusters. 14:31:24.335 [D] model:569 No channel shank file found. 14:31:24.336 [D] model:692 Loading templates. 14:31:24.337 [D] model:724 Templates are dense. 14:31:24.337 [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed. 14:31:24.337 [D] model:730 Loading the whitening matrix. 14:31:24.337 [D] model:434 Whitening matrix file not found. 14:31:24.342 [D] model:737 Loading the inverse of the whitening matrix. 14:31:24.342 [D] model:440 Whitening matrix inverse file not found, computing it. 14:31:24.342 [D] model:744 Inversing the whitening matrix (46, 46). 14:31:24.404 [W] traces:462 File C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\None does not exist. 14:31:24.404 [W] traces:462 File C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\None does not exist. 14:31:24.404 [D] model:766 Loading features. 14:31:24.404 [D] model:788 Features are dense. 14:31:24.404 [D] model:803 Loading template features. 14:31:24.902 [D] context:80 Create cache directory C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy. 14:31:25.002 [D] context:100 Initialize joblib cache dir at C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy. 14:31:25.018 [D] context:101 Reducing the size of the cache if needed. 14:31:25.021 [D] base:102 Add filter high_pass. 14:31:25.024 [D] config:31 Load config file C:\Users\guoyu\.phy\phy_config.py. 14:31:25.024 [D] plugin:145 Loading 0 plugins. 14:31:25.037 [D] context:209 The file C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy\new_cluster_id.pkl doesn't exist. 14:31:25.043 [D] context:209 The file C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy\spikes_per_cluster.pkl doesn't exist. 14:31:25.043 [D] clustering:237 Recompute spikes_per_cluster manually: this might take a while. 14:31:25.043 [D] context:185 Save data to C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy\spikes_per_cluster.pkl. 14:31:25.052 [D] gui:463 Creating GUI. 14:31:25.119 [D] state:46 Load C:\Users\guoyu.phy\TemplateGUI\state.json for GUIState. 14:31:27.195 [D] gui:718 Add view ClusterView to GUI. 14:31:27.246 [D] gui:718 Add view SimilarityView to GUI. 14:31:27.317 [D] gui:718 Add view WaveformView to GUI. 14:31:27.336 [D] base:337 Set state for WaveformView. 14:31:27.355 [D] gui:718 Add view CorrelogramView to GUI. 14:31:27.360 [D] base:337 Set state for CorrelogramView. 14:31:27.396 [D] gui:718 Add view ISIView to GUI. 14:31:27.398 [D] base:337 Set state for ISIView. 14:31:27.426 [D] gui:718 Add view FeatureView to GUI. 14:31:27.429 [D] base:337 Set state for FeatureView. 14:31:27.475 [D] gui:718 Add view AmplitudeView to GUI. 14:31:27.477 [D] base:337 Set state for AmplitudeView. 14:31:27.509 [D] gui:718 Add view FiringRateView to GUI. 14:31:27.512 [D] base:337 Set state for FiringRateView. 14:31:27.514 [W] gui:671 Could not create view TraceView. 14:31:27.539 [D] gui:718 Add view ProbeView to GUI. 14:31:27.542 [D] base:337 Set state for ProbeView. 14:31:27.543 [W] gui:671 Could not create view TemplateFeatureView. 14:31:27.573 [D] gui:718 Add view ClusterScatterView to GUI. 14:31:27.577 [D] base:337 Set state for ClusterScatterView. 14:31:27.577 [D] base:494 Set color scheme to firing_rate. 14:31:27.579 [D] base:1626 Adding default color schemes to ClusterScatterView. 14:31:28.231 [D] gui:519 Load the geometry state. QMimeDatabase: Error loading internal MIME data An error has been encountered at line 1 of : Premature end of document.: 14:31:35.778 [W] gui:671 Could not create view TemplateFeatureView.

Folder: image

zm711 commented 4 months ago

Hey Florent,

How did you install Phy? You have the QMimeDatabase error which is often due to some install problems.

florian6973 commented 4 months ago

Hi Zach!

Thanks for your reply! I strictly followed the current readme, creating a conda environment with python 3.11, then pip install from git and finally opening phy...

zm711 commented 4 months ago

I think there might be an issue with conda-forge for one of the packages. I haven't diagnosed which one yet.

could you make do the following (after cloning the repo. You'll either need to edit the requirements.txt to numpy<2.0 or go from my recent PR #1289.

conda create -n phy2_test python=3.11
cd phy
pip install -e .

And see if that works. I'm still trying to diagnose why installing purely with the conda instructions doesn't seem to be working since NumPy 2.0 came out.

florian6973 commented 4 months ago

Thanks for your reply! After following this new installation workflow, it still does not work unfortunately (without the MIME error though now):

16:17:34.280 [D] init:68 Start capturing exceptions. 16:17:34.280 [W] model:1415 File None does not exist. 16:17:34.296 [D] model:619 Loading spike clusters. 16:17:34.296 [D] model:569 No channel shank file found. 16:17:34.296 [D] model:692 Loading templates. 16:17:34.296 [D] model:724 Templates are dense. 16:17:34.296 [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed. 16:17:34.296 [D] model:730 Loading the whitening matrix. 16:17:34.296 [D] model:434 Whitening matrix file not found. 16:17:34.296 [D] model:737 Loading the inverse of the whitening matrix. 16:17:34.296 [W] traces:462 File C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\None does not exist. 16:17:34.296 [W] traces:462 File C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\None does not exist. 16:17:34.296 [D] model:766 Loading features. 16:17:34.312 [D] model:788 Features are dense. 16:17:34.312 [D] model:803 Loading template features. 16:17:34.878 [D] context:100 Initialize joblib cache dir at C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy. 16:17:34.878 [D] context:101 Reducing the size of the cache if needed. 16:17:34.894 [D] base:102 Add filter high_pass. 16:17:34.894 [D] config:31 Load config file C:\Users\guoyu\.phy\phy_config.py. 16:17:34.894 [D] plugin:146 Loading 0 plugins. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base._get_template_waveforms. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base.get_mean_spike_template_amplitudes. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base.get_template_counts. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base.get_template_for_cluster. 16:17:34.894 [D] context:126 Load memcache for phy.apps.template.gui.get_template_amplitude. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base.get_cluster_amplitude. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base.get_mean_firing_rate. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base.get_best_channel. 16:17:34.894 [D] context:126 Load memcache for phy.apps.template.gui.get_best_channels. 16:17:34.894 [D] context:126 Load memcache for phy.apps.base.get_channel_shank. 16:17:34.909 [D] context:126 Load memcache for phy.apps.base.get_probe_depth. 16:17:34.911 [D] context:126 Load memcache for phy.apps.base.peak_channel_similarity. 16:17:34.912 [D] context:209 The file C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy\new_cluster_id.pkl doesn't exist. 16:17:34.925 [D] context:185 Save data to C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy\.phy\spikes_per_cluster.pkl. 16:17:34.928 [D] gui:463 Creating GUI. 16:17:34.988 [D] state:46 Load C:\Users\guoyu.phy\TemplateGUI\state.json for GUIState. 16:17:34.988 [D] state:46 Load C:\Users\guoyu\OneDrive\Documents\Florent\SpikeSorting\ms5-phy\phy.phy\state.json for GUIState. 16:17:36.326 [D] gui:718 Add view ClusterView to GUI. 16:17:36.360 [D] gui:718 Add view SimilarityView to GUI. 16:17:36.482 [D] gui:718 Add view WaveformView to GUI. 16:17:36.496 [D] base:337 Set state for WaveformView. 16:17:36.514 [D] gui:718 Add view CorrelogramView to GUI. 16:17:36.517 [D] base:337 Set state for CorrelogramView. 16:17:36.539 [D] gui:718 Add view ISIView to GUI. 16:17:36.541 [D] base:337 Set state for ISIView. 16:17:36.602 [D] gui:718 Add view FeatureView to GUI. 16:17:36.606 [D] base:337 Set state for FeatureView. 16:17:36.648 [D] gui:718 Add view AmplitudeView to GUI. 16:17:36.651 [D] base:337 Set state for AmplitudeView. 16:17:36.672 [D] gui:718 Add view FiringRateView to GUI. 16:17:36.674 [D] base:337 Set state for FiringRateView. 16:17:36.691 [D] gui:718 Add view ProbeView to GUI. 16:17:36.694 [D] base:337 Set state for ProbeView. 16:17:36.720 [D] gui:718 Add view ClusterScatterView to GUI. 16:17:36.722 [D] base:337 Set state for ClusterScatterView. 16:17:36.722 [D] base:494 Set color scheme to random. 16:17:36.723 [D] base:1626 Adding default color schemes to ClusterScatterView. 16:17:37.454 [D] gui:519 Load the geometry state. 16:17:39.273 [D] supervisor:817 Clusters selected: [6] (5) 16:17:43.619 [W] gui:671 Could not create view TemplateFeatureView.

zm711 commented 4 months ago

Why is your file None?

16:17:34.280 [W] model:1415 File None does not exist.
florian6973 commented 4 months ago

I exported to phy with these options to minimize the size: si.export_to_phy(waveform_extractor, output_folder='phy', verbose=True, use_relative_path=True, copy_binary=False) Since the PC features were properly showing (see below), I did think it would be the same for the template features.

image

So I exported again with copy_binary=True (now phy can access the recording), but the TemplateFeatureView still does not show up:

(phy2_test) C:\Users\guoyu\Downloads\phy2>phy template-gui params.py --debug 08:02:30.897 [D] init:68 Start capturing exceptions. 08:02:30.944 [D] model:619 Loading spike clusters. 08:02:30.991 [D] model:569 No channel shank file found. 08:02:30.991 [D] model:692 Loading templates. 08:02:30.991 [D] model:724 Templates are dense. 08:02:31.007 [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed. 08:02:31.007 [D] model:730 Loading the whitening matrix. 08:02:31.007 [D] model:434 Whitening matrix file not found. 08:02:31.007 [D] model:737 Loading the inverse of the whitening matrix. 08:02:31.007 [D] model:440 Whitening matrix inverse file not found, computing it. 08:02:31.007 [D] model:744 Inversing the whitening matrix (1, 1). 08:02:31.024 [D] model:766 Loading features. 08:02:31.043 [D] model:781 Features are sparse. 08:02:31.043 [D] model:803 Loading template features. 08:02:31.043 [D] model:504 Load cluster_amplitude_cutoff.tsv. 08:02:31.053 [D] model:504 Load cluster_amplitude_cv_median.tsv. 08:02:31.060 [D] model:504 Load cluster_amplitude_cv_range.tsv. 08:02:31.071 [D] model:504 Load cluster_amplitude_median.tsv. 08:02:31.071 [D] model:504 Load cluster_channel_group.tsv. 08:02:31.086 [D] model:504 Load cluster_firing_range.tsv. 08:02:31.101 [D] model:504 Load cluster_group.tsv. 08:02:31.109 [D] model:504 Load cluster_isi_violations_count.tsv. 08:02:31.117 [D] model:504 Load cluster_isi_violations_ratio.tsv. 08:02:31.133 [D] model:504 Load cluster_presence_ratio.tsv. 08:02:31.143 [D] model:504 Load cluster_rp_contamination.tsv. 08:02:31.157 [D] model:504 Load cluster_rp_violations.tsv. 08:02:31.165 [D] model:504 Load cluster_sd_ratio.tsv. 08:02:31.182 [D] model:504 Load cluster_si_unit_ids.tsv. 08:02:31.192 [D] model:504 Load cluster_sliding_rp_violation.tsv. 08:02:31.197 [D] model:504 Load cluster_snr.tsv. 08:02:31.213 [D] model:504 Load cluster_sync_spike_2.tsv. 08:02:31.213 [D] model:504 Load cluster_sync_spike_4.tsv. 08:02:31.229 [D] model:504 Load cluster_sync_spike_8.tsv. 08:02:31.711 [D] context:80 Create cache directory C:\Users\guoyu\Downloads\phy2\.phy. 08:02:31.792 [D] context:100 Initialize joblib cache dir at C:\Users\guoyu\Downloads\phy2\.phy. 08:02:31.792 [D] context:101 Reducing the size of the cache if needed. 08:02:31.792 [D] base:102 Add filter high_pass. 08:02:31.792 [D] config:31 Load config file C:\Users\guoyu\.phy\phy_config.py. 08:02:31.810 [D] plugin:146 Loading 0 plugins. 08:02:31.822 [D] context:209 The file C:\Users\guoyu\Downloads\phy2\.phy\new_cluster_id.pkl doesn't exist. 08:02:31.822 [D] context:209 The file C:\Users\guoyu\Downloads\phy2\.phy\spikes_per_cluster.pkl doesn't exist. 08:02:31.822 [D] clustering:237 Recompute spikes_per_cluster manually: this might take a while. 08:02:31.822 [D] context:185 Save data to C:\Users\guoyu\Downloads\phy2\.phy\spikes_per_cluster.pkl. 08:02:31.822 [D] gui:463 Creating GUI. 08:02:31.869 [D] state:46 Load C:\Users\guoyu.phy\TemplateGUI\state.json for GUIState. 08:02:33.456 [D] gui:718 Add view ClusterView to GUI. 08:02:33.490 [D] gui:718 Add view SimilarityView to GUI. 08:02:33.702 [D] gui:718 Add view WaveformView to GUI. 08:02:33.718 [D] base:337 Set state for WaveformView. 08:02:33.749 [D] gui:718 Add view CorrelogramView to GUI. 08:02:33.753 [D] base:337 Set state for CorrelogramView. 08:02:33.803 [D] gui:718 Add view ISIView to GUI. 08:02:33.809 [D] base:337 Set state for ISIView. 08:02:33.882 [D] gui:718 Add view FeatureView to GUI. 08:02:33.888 [D] base:337 Set state for FeatureView. 08:02:33.950 [D] gui:718 Add view AmplitudeView to GUI. 08:02:33.955 [D] base:337 Set state for AmplitudeView. 08:02:33.999 [D] gui:718 Add view FiringRateView to GUI. 08:02:34.002 [D] base:337 Set state for FiringRateView. 08:02:34.020 [D] gui:718 Add view ProbeView to GUI. 08:02:34.022 [D] base:337 Set state for ProbeView. 08:02:34.061 [D] gui:718 Add view ClusterScatterView to GUI. 08:02:34.066 [D] base:337 Set state for ClusterScatterView. 08:02:34.067 [D] base:494 Set color scheme to random. 08:02:34.068 [D] base:1626 Adding default color schemes to ClusterScatterView. 08:02:34.113 [D] gui:718 Add view TemplateView to GUI. 08:02:34.116 [D] base:337 Set state for TemplateView. 08:02:34.117 [D] base:494 Set color scheme to random. 08:02:34.118 [D] base:1626 Adding default color schemes to TemplateView. 08:02:34.155 [D] gui:718 Add view TemplateView to GUI. 08:02:34.160 [D] base:337 Set state for TemplateView (1). 08:02:34.161 [D] base:494 Set color scheme to random. 08:02:34.163 [D] base:1626 Adding default color schemes to TemplateView (1). 08:02:34.912 [D] gui:519 Load the geometry state. 08:02:39.395 [D] base:276 Close view TemplateView (1). 08:02:39.395 [D] state:171 Update GUI state for TemplateView (1) 08:02:40.870 [D] supervisor:817 Clusters selected: [0] (4) 08:02:41.856 [D] supervisor:839 Similar clusters selected: [7] (4) 08:02:45.576 [W] gui:671 Could not create view TemplateFeatureView.

zm711 commented 4 months ago

Hey Florent, could you list all files in the folder. Either as a full picture or just as a list. This could be that the export_to_phy function is missing something. We've added some additional features to it with the 0.101.0 release coming soon, but I know you're trying to stay on 0.100.x for now. Which sorter were you using?

florian6973 commented 3 months ago

For this one I was using MountainSort5, here are all the files in the folder:

Folder PATH listing for volume Windows-SSD Volume serial number is 340B-6FD7 C:. │ amplitudes.npy │ channel_groups.npy │ channel_map.npy │ channel_map_si.npy │ channel_positions.npy │ cluster_amplitude_cutoff.tsv │ cluster_amplitude_cv_median.tsv │ cluster_amplitude_cv_range.tsv │ cluster_amplitude_median.tsv │ cluster_channel_group.tsv │ cluster_firing_range.tsv │ cluster_group.tsv │ cluster_isi_violations_count.tsv │ cluster_isi_violations_ratio.tsv │ cluster_presence_ratio.tsv │ cluster_rp_contamination.tsv │ cluster_rp_violations.tsv │ cluster_sd_ratio.tsv │ cluster_si_unit_ids.tsv │ cluster_sliding_rp_violation.tsv │ cluster_snr.tsv │ cluster_sync_spike_2.tsv │ cluster_sync_spike_4.tsv │ cluster_sync_spike_8.tsv │ params.py │ pc_features.npy │ pc_feature_ind.npy │ phy.log │ recording.dat │ similar_templates.npy │ spike_clusters.npy │ spike_templates.npy │ spike_times.npy │ templates.npy │ whitening_mat_inv.npy │ └───.phy │ spikes_per_cluster.pkl │ state.json │ ├───memcache │ phy.apps.base.get_best_channel.pkl │ phy.apps.base.get_channel_shank.pkl │ phy.apps.base.get_cluster_amplitude.pkl │ phy.apps.base.get_mean_firing_rate.pkl │ phy.apps.base.get_mean_spike_template_amplitudes.pkl │ phy.apps.base.get_probe_depth.pkl │ phy.apps.base.get_template_counts.pkl │ phy.apps.base.get_template_for_cluster.pkl │ phy.apps.base.peak_channel_similarity.pkl │ phy.apps.base._get_mean_waveforms.pkl │ phy.apps.base._get_template_waveforms.pkl │ phy.apps.template.gui.get_best_channels.pkl │ phy.apps.template.gui.get_template_amplitude.pkl │ └───phy └───apps └───base ├───BaseController │ ├───_get_correlograms │ │ │ func_code.py │ │ │ │ │ ├───9440f18609f2fb2f54002db34d535c28 │ │ │ metadata.json │ │ │ output.pkl │ │ │ │ │ └───e9380640d7a1fce9450e896212c3c529 │ │ metadata.json │ │ output.pkl │ │ │ └───_get_correlograms_rate │ │ func_code.py │ │ │ ├───21c011882470d608d668678de26c8a58 │ │ metadata.json │ │ output.pkl │ │ │ └───78ac1302226b9c5bf26c5633cab6fb29 │ metadata.json │ output.pkl │ ├───FeatureMixin │ ├───get_spike_feature_amplitudes │ └───_get_features │ │ func_code.py │ │ │ ├───24e3400218971593519fa9a3675b4ec8 │ │ metadata.json │ │ output.pkl │ │ │ ├───7a48f028123fd49893a06fc6ef2129f3 │ │ metadata.json │ │ output.pkl │ │ │ └───e8f9fb7c0b150b4269a23168aca1e0be │ metadata.json │ output.pkl │ ├───TemplateMixin │ ├───get_amplitudes │ ├───get_spike_template_amplitudes │ │ │ func_code.py │ │ │ │ │ ├───0fe0be0053102709521f1c4d56c35e9b │ │ │ metadata.json │ │ │ output.pkl │ │ │ │ │ ├───60d8a3e5695717826d97c51421f4c9aa │ │ │ metadata.json │ │ │ output.pkl │ │ │ │ │ ├───a887b0a772bdda52b283dffbc81cf48e │ │ │ metadata.json │ │ │ output.pkl │ │ │ │ │ └───adbdce6361b9abc70d4b39502596cdbd │ │ metadata.json │ │ output.pkl │ │ │ └───get_spike_template_features └───WaveformMixin └───_get_waveforms_with_n_spikes image

If you think it should be solved with 0.101.0, I can test it as well, and if so it will strongly motivate a faster transition to this new version...

florian6973 commented 3 months ago

Just tested 0.101.0rc0, and I get the same error unfortunately:

rec_si, _ = si.read_mearec(path)
sort_ms = si.run_sorter("mountainsort5", rec_si, remove_existing_folder=True)
sa = si.create_sorting_analyzer(sort_ms, rec_si)
sa.compute(['random_spikes'])
sa.compute(['waveforms'])
sa.compute(['templates'])
si.export_to_phy(
    sorting_analyzer=sa,
    output_folder='phy_test_2',
    use_relative_path=True,    
)

10:02:22.629 [D] init:68 Start capturing exceptions. 10:02:22.636 [D] model:619 Loading spike clusters. 10:02:22.641 [D] model:569 No channel shank file found. 10:02:22.643 [D] model:692 Loading templates. 10:02:22.647 [D] model:720 Templates are sparse. 10:02:22.648 [W] model:667 Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed. 10:02:22.649 [D] model:730 Loading the whitening matrix. 10:02:22.649 [D] model:434 Whitening matrix file not found. 10:02:22.649 [D] model:737 Loading the inverse of the whitening matrix. 10:02:22.652 [D] model:766 Loading features. 10:02:22.653 [D] model:781 Features are sparse. 10:02:22.653 [D] model:803 Loading template features. 10:02:22.654 [D] model:504 Load cluster_channel_group.tsv. 10:02:22.655 [D] model:504 Load cluster_group.tsv. 10:02:22.656 [D] model:504 Load cluster_si_unit_ids.tsv. 10:02:23.128 [D] context:100 Initialize joblib cache dir at C:\Users\guoyu\Downloads\phy_test_2\.phy. 10:02:23.128 [D] context:101 Reducing the size of the cache if needed. 10:02:23.141 [D] base:102 Add filter high_pass. 10:02:23.144 [D] config:31 Load config file C:\Users\guoyu\.phy\phy_config.py. 10:02:23.146 [D] plugin:146 Loading 0 plugins. 10:02:23.147 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms. 10:02:23.148 [D] context:126 Load memcache for phy.apps.base._get_mean_waveforms. 10:02:23.150 [D] context:126 Load memcache for phy.apps.base._get_template_waveforms. 10:02:23.152 [D] context:126 Load memcache for phy.apps.base.get_mean_spike_template_amplitudes. 10:02:23.153 [D] context:126 Load memcache for phy.apps.base.get_template_counts. 10:02:23.154 [D] context:126 Load memcache for phy.apps.base.get_template_for_cluster. 10:02:23.155 [D] context:126 Load memcache for phy.apps.template.gui.get_template_amplitude. 10:02:23.156 [D] context:126 Load memcache for phy.apps.base.get_cluster_amplitude. 10:02:23.159 [D] context:126 Load memcache for phy.apps.base.get_mean_firing_rate. 10:02:23.161 [D] context:126 Load memcache for phy.apps.base.get_best_channel. 10:02:23.161 [D] context:126 Load memcache for phy.apps.template.gui.get_best_channels. 10:02:23.163 [D] context:126 Load memcache for phy.apps.base.get_channel_shank. 10:02:23.164 [D] context:126 Load memcache for phy.apps.base.get_probe_depth. 10:02:23.166 [D] context:126 Load memcache for phy.apps.base.peak_channel_similarity. 10:02:23.169 [D] context:209 The file C:\Users\guoyu\Downloads\phy_test_2\.phy\new_cluster_id.pkl doesn't exist. 10:02:23.186 [D] context:185 Save data to C:\Users\guoyu\Downloads\phy_test_2\.phy\spikes_per_cluster.pkl. 10:02:23.196 [D] gui:463 Creating GUI. 10:02:23.245 [D] state:46 Load C:\Users\guoyu.phy\TemplateGUI\state.json for GUIState. 10:02:23.248 [D] state:46 Load C:\Users\guoyu\Downloads\phy_test_2.phy\state.json for GUIState. 10:02:24.529 [D] gui:718 Add view ClusterView to GUI. 10:02:24.561 [D] gui:718 Add view SimilarityView to GUI. 10:02:24.682 [D] gui:718 Add view WaveformView to GUI. 10:02:24.696 [D] base:337 Set state for WaveformView. 10:02:24.726 [D] gui:718 Add view CorrelogramView to GUI. 10:02:24.732 [D] base:337 Set state for CorrelogramView. 10:02:24.788 [D] gui:718 Add view ISIView to GUI. 10:02:24.798 [D] base:337 Set state for ISIView. 10:02:24.828 [D] gui:718 Add view FeatureView to GUI. 10:02:24.831 [D] base:337 Set state for FeatureView. 10:02:24.879 [D] gui:718 Add view AmplitudeView to GUI. 10:02:24.884 [D] base:337 Set state for AmplitudeView. 10:02:24.931 [D] gui:718 Add view FiringRateView to GUI. 10:02:24.938 [D] base:337 Set state for FiringRateView. 10:02:24.971 [D] gui:718 Add view ProbeView to GUI. 10:02:24.979 [D] base:337 Set state for ProbeView. 10:02:25.018 [D] gui:718 Add view ClusterScatterView to GUI. 10:02:25.023 [D] base:337 Set state for ClusterScatterView. 10:02:25.023 [D] base:494 Set color scheme to random. 10:02:25.030 [D] base:1626 Adding default color schemes to ClusterScatterView. 10:02:25.083 [D] gui:718 Add view TemplateView to GUI. 10:02:25.087 [D] base:337 Set state for TemplateView. 10:02:25.087 [D] base:494 Set color scheme to random. 10:02:25.088 [D] base:1626 Adding default color schemes to TemplateView. 10:02:26.188 [D] gui:519 Load the geometry state. 10:02:28.457 [D] supervisor:817 Clusters selected: [6] (8) 10:02:30.592 [W] gui:671 Could not create view TemplateFeatureView. 10:03:23.504 [D] supervisor:839 Similar clusters selected: [3] (7)

I also heard that Sam plans to release spikeinterface-gui with curation features, do you think it is as mature as phy regarding functionalities please? Thanks

image

zm711 commented 3 months ago

It looks like you are missing template_feature_ind and templates_features. Could you check for those? This could be a problem with our export to phy function in si. If it is a problem over there then we need to check with Sam and Alessio to see if there is a reason we can't generate those files (ie we can't make the whitening matrix because not every sorter does whitening) or we just haven't yet.

florian6973 commented 3 months ago

Thanks for your reply! I checked but I could not find them unfortunately. It seems these files are not created in https://github.com/SpikeInterface/spikeinterface/blob/1197aad12b7170432eec8b887e24b9bc29921eab/src/spikeinterface/exporters/to_phy.py ... Should I open a spikeinterface issue then?

zm711 commented 3 months ago

Yeah I would open the issue there. That way you can ask if it was intentional not to make those files :)