Waveforms view only show mean waveforms

[Antho2422]

Hi all, I have an issue using the waveforms view. I can only display the mean waveform view and the button "toggle mean waveforms" does not have any effect.

Here is my phy folder : https://instituteicm-my.sharepoint.com/:f:/g/personal/anthony_pinto_icm-institute_org/EjKkyzBR4O5OvbYDfxM1R9sBHFJO2Mmn3hOWhgyv42xfYg?e=jwBj4v

Here is the log file if you want to have a quick look at it before downloading all the data.

16:13:31.396 [D] context:80           Create cache directory `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.396 [D] context:80           Create cache directory `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.806 [D] context:100          Initialize joblib cache dir at `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.806 [D] context:100          Initialize joblib cache dir at `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.826 [D] context:101          Reducing the size of the cache if needed.
16:13:31.826 [D] context:101          Reducing the size of the cache if needed.
16:13:32.017 [D] base:102             Add filter `high_pass`.
16:13:32.017 [D] base:102             Add filter `high_pass`.
16:13:32.101 [D] config:31            Load config file `C:\Users\anthony.pinto\.phy\phy_config.py`.
16:13:32.101 [D] config:31            Load config file `C:\Users\anthony.pinto\.phy\phy_config.py`.
16:13:32.189 [D] plugin:145           Loading 0 plugins.
16:13:32.189 [D] plugin:145           Loading 0 plugins.
16:13:32.866 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\new_cluster_id.pkl` doesn't exist.
16:13:32.866 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\new_cluster_id.pkl` doesn't exist.
16:13:32.888 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl` doesn't exist.
16:13:32.888 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl` doesn't exist.
16:13:32.901 [D] clustering:237       Recompute spikes_per_cluster manually: this might take a while.
16:13:32.901 [D] clustering:237       Recompute spikes_per_cluster manually: this might take a while.
16:13:32.974 [D] context:185          Save data to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl`.
16:13:32.974 [D] context:185          Save data to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl`.
16:13:33.257 [D] gui:463              Creating GUI.
16:13:33.257 [D] gui:463              Creating GUI.
16:13:33.333 [D] state:46             Load C:\Users\anthony.pinto\.phy\TemplateGUI\state.json for GUIState.
16:13:33.333 [D] state:46             Load C:\Users\anthony.pinto\.phy\TemplateGUI\state.json for GUIState.
16:13:34.035 [D] gui:718              Add view ClusterView to GUI.
16:13:34.035 [D] gui:718              Add view ClusterView to GUI.
16:13:34.186 [D] gui:718              Add view SimilarityView to GUI.
16:13:34.186 [D] gui:718              Add view SimilarityView to GUI.
16:13:34.293 [D] gui:718              Add view WaveformView to GUI.
16:13:34.293 [D] gui:718              Add view WaveformView to GUI.
16:13:34.372 [D] base:337             Set state for WaveformView.
16:13:34.372 [D] base:337             Set state for WaveformView.
16:13:34.465 [D] gui:718              Add view CorrelogramView to GUI.
16:13:34.465 [D] gui:718              Add view CorrelogramView to GUI.
16:13:34.536 [D] base:337             Set state for CorrelogramView.
16:13:34.536 [D] base:337             Set state for CorrelogramView.
16:13:34.623 [D] gui:718              Add view ISIView to GUI.
16:13:34.623 [D] gui:718              Add view ISIView to GUI.
16:13:34.693 [D] base:337             Set state for ISIView.
16:13:34.693 [D] base:337             Set state for ISIView.
16:13:34.784 [D] gui:718              Add view FeatureView to GUI.
16:13:34.784 [D] gui:718              Add view FeatureView to GUI.
16:13:34.856 [D] base:337             Set state for FeatureView.
16:13:34.856 [D] base:337             Set state for FeatureView.
16:13:34.954 [D] gui:718              Add view AmplitudeView to GUI.
16:13:34.954 [D] gui:718              Add view AmplitudeView to GUI.
16:13:35.074 [D] base:337             Set state for AmplitudeView.
16:13:35.074 [D] base:337             Set state for AmplitudeView.
16:13:35.111 [D] gui:718              Add view FiringRateView to GUI.
16:13:35.111 [D] gui:718              Add view FiringRateView to GUI.
16:13:35.125 [D] base:337             Set state for FiringRateView.
16:13:35.125 [D] base:337             Set state for FiringRateView.
16:13:35.444 [D] gui:519              Load the geometry state.
16:13:35.444 [D] gui:519              Load the geometry state.
16:13:38.599 [D] supervisor:817       Clusters selected: [13] (20)
16:13:38.599 [D] supervisor:817       Clusters selected: [13] (20)
16:16:12.074 [D] state:171            Update GUI state for ClusterView
16:16:12.074 [D] state:171            Update GUI state for ClusterView
16:16:12.139 [D] state:171            Update GUI state for WaveformView
16:16:12.139 [D] state:171            Update GUI state for WaveformView
16:16:12.205 [D] state:171            Update GUI state for CorrelogramView
16:16:12.205 [D] state:171            Update GUI state for CorrelogramView
16:16:12.268 [D] state:171            Update GUI state for ISIView
16:16:12.268 [D] state:171            Update GUI state for ISIView
16:16:12.387 [D] state:171            Update GUI state for FeatureView
16:16:12.387 [D] state:171            Update GUI state for FeatureView
16:16:12.428 [D] state:171            Update GUI state for AmplitudeView
16:16:12.428 [D] state:171            Update GUI state for AmplitudeView
16:16:12.448 [D] state:171            Update GUI state for FiringRateView
16:16:12.448 [D] state:171            Update GUI state for FiringRateView
16:16:12.455 [D] context:138          Save memcache for `phy.apps.base._get_mean_waveforms`.
16:16:12.455 [D] context:138          Save memcache for `phy.apps.base._get_mean_waveforms`.
16:16:12.494 [D] context:138          Save memcache for `phy.apps.base._get_template_waveforms`.
16:16:12.494 [D] context:138          Save memcache for `phy.apps.base._get_template_waveforms`.
16:16:12.527 [D] context:138          Save memcache for `phy.apps.base.get_mean_spike_template_amplitudes`.
16:16:12.527 [D] context:138          Save memcache for `phy.apps.base.get_mean_spike_template_amplitudes`.
16:16:12.562 [D] context:138          Save memcache for `phy.apps.base.get_template_counts`.
16:16:12.562 [D] context:138          Save memcache for `phy.apps.base.get_template_counts`.
16:16:12.601 [D] context:138          Save memcache for `phy.apps.base.get_template_for_cluster`.
16:16:12.601 [D] context:138          Save memcache for `phy.apps.base.get_template_for_cluster`.
16:16:12.634 [D] context:138          Save memcache for `phy.apps.template.gui.get_template_amplitude`.
16:16:12.634 [D] context:138          Save memcache for `phy.apps.template.gui.get_template_amplitude`.
16:16:12.742 [D] context:138          Save memcache for `phy.apps.base.get_cluster_amplitude`.
16:16:12.742 [D] context:138          Save memcache for `phy.apps.base.get_cluster_amplitude`.
16:16:12.837 [D] context:138          Save memcache for `phy.apps.base.get_mean_firing_rate`.
16:16:12.837 [D] context:138          Save memcache for `phy.apps.base.get_mean_firing_rate`.
16:16:12.912 [D] context:138          Save memcache for `phy.apps.base.get_best_channel`.
16:16:12.912 [D] context:138          Save memcache for `phy.apps.base.get_best_channel`.
16:16:12.967 [D] context:138          Save memcache for `phy.apps.template.gui.get_best_channels`.
16:16:12.967 [D] context:138          Save memcache for `phy.apps.template.gui.get_best_channels`.
16:16:13.030 [D] context:138          Save memcache for `phy.apps.base.get_channel_shank`.
16:16:13.030 [D] context:138          Save memcache for `phy.apps.base.get_channel_shank`.
16:16:13.084 [D] context:138          Save memcache for `phy.apps.base.get_probe_depth`.
16:16:13.084 [D] context:138          Save memcache for `phy.apps.base.get_probe_depth`.
16:16:13.139 [D] context:138          Save memcache for `phy.apps.base.peak_channel_similarity`.
16:16:13.139 [D] context:138          Save memcache for `phy.apps.base.peak_channel_similarity`.
16:16:13.229 [D] gui:600              Save the geometry state.
16:16:13.229 [D] gui:600              Save the geometry state.
16:16:13.240 [D] state:218            Save global GUI state to `C:\Users\anthony.pinto\.phy\TemplateGUI\state.json`.
16:16:13.240 [D] state:218            Save global GUI state to `C:\Users\anthony.pinto\.phy\TemplateGUI\state.json`.
16:16:13.256 [D] state:232            Save local GUI state to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\state.json`.
16:16:13.256 [D] state:232            Save local GUI state to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\state.json`.

I am a bit confused about the exact reason of this issue. I exported the data using the function from spikeinterface like that:

sexp.export_to_phy(
                waveform_extractor=waveforms,
                output_folder=os.path.join(output_folder, 'phy'),
                compute_pc_features=True,
                compute_amplitudes=True,
                copy_binary=True,
                peak_sign='neg',
                remove_if_exists=True,
                verbose=True,
            )

To add a bit more context, my folder is located at a very long path and it was the origin of a problem with the waveforms a few weeks ago as mentionned here https://github.com/cortex-lab/phy/issues/1219#issue-1880538192 but I found a workaround by doing that : https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/ so I avoided the windows limit of 260 characters. I though it solved my issue because with the previous issue the waveforms were not displaying at all and now I can see them. I would like to find a workaround because I would like to avoid to move the location of the phy folder if possible.

Cheers, Anthony

[zm711]

@Antho2422,

Hey Anthony, happy we got through one issue hopefully we can figure this out. It looks like we are missing the beginning of the log since it should say when it is loading files. (It should say whether templates are sparse or dense etc). Could you include that in the log as well. Did you run

phy template-gui params.py --debug

I just want to make sure we have the full log, so we can see why it is doing the mean only and not raw. Unfortunately I'm pretty busy for the next week, but I'm happy to troubleshoot more next week.

[Antho2422]

@zm711

Thank you for your answer. I will run the debug mode asap but I'm pretty sure the issue is the same as the last one. It happens only when the path to my phy folder is really long. I was wondering if maybe we can allow relative path.

Would be very usefull because I'm using phy in a pipeline where the architecture of folders is organized in a certain way to run multiple experiment and facilitate the user usage and I would like to keep it like that. I would like to be able to launch phy from a folder without having to move the folder before opening it.

If we find a solution for that I would be more than happy 👍

Thank you very much ! Cheers

[zm711]

@Antho2422,

After you last issue I added that option to Spikeinterface :)

Do:

exp.export_to_phy(
                waveform_extractor=waveforms,
                output_folder=os.path.join(output_folder, 'phy'),
                compute_pc_features=True,
                compute_amplitudes=True,
                copy_binary=True,
                peak_sign='neg',
                remove_if_exists=True,
                verbose=True,
                use_relative_path=True # here is the new argument
            )

It will require the newest version of the SpikeInterface (0.99.1 with pip install or install from source). Give it a try and let me know if that helps. Zach

[Antho2422]

@zm711

Oh I did not notice that thank you ! I will check as soon as possible

[meszena]

Hi! I have the same problem, in WaveformView the raw display works only, I am not able to switch to "Toggle Mean Waveforms" either using "M" or the drop-down menu with the mouse.

Have you found a solution for this? I don't have as long path names so that part should be OK. Thanks in advance!

[allllex3202]

Hi, I have the same issue. I want to double-check a file I've sorted, but was only able to see the mean waveforms the another day, no buttons works to update this view. After I restart the Win11 computer, the whole waveforms disappears... no mean waveforms even.

I don't think I changed any settings across the 3 opening mentioned above (1st normal, 2nd only means, 3rd time nothing). Does anyone have a clue?

Thanks!

[allllex3202]

Hi, I have the same issue. I want to double-check a file I've sorted, but was only able to see the mean waveforms the another day, no buttons works to update this view. After I restart the Win11 computer, the whole waveforms disappears... no mean waveforms even.

I don't think I changed any settings across the 3 opening mentioned above (1st normal, 2nd only means, 3rd time nothing). Does anyone have a clue?

Thanks!

Update, close the default view and open a new waveform windows recovers the display issue.