SpikeInterface / spikeinterface

A Python-based module for creating flexible and robust spike sorting pipelines.
https://spikeinterface.readthedocs.io
MIT License
519 stars 186 forks source link

The waveforms are upside down. #3492

Open BEGINRX opened 5 days ago

BEGINRX commented 5 days ago

I can't find the reason of this problem. The waveforms in the Phy are normal, but the waveforms are upside down when the results are imported into the sorting analyzer. 图片1 image

zm711 commented 5 days ago

Could you let us know which recording you are using? Maybe a little bit of the script you ran to make your sorting analyzer. If the gain has the wrong sign this could happen for sure. So it would help us to see what you did to get to this point :)

alejoe91 commented 5 days ago

Yes I think you might not be looking at the same channel. How are you generating the spikeinterface plot?

BEGINRX commented 5 days ago

Sorry. I will add my code here. First, I export the sorting analyzer. export_to_phy(sa, phy_res_path, compute_amplitudes=True, compute_pc_features=True, remove_if_exists=True) Then, I read the data from the result.

    info_path = os.path.join(phy_folder, 'reportinfo.json')
    file = open(info_path, 'r')
    params = json.load(file)
    peak_sign = params['peak_sign']
    radius_um = params['radius_um']
    file_names = params['file_names']
    frame_list = params['frame_list']
    channel_ids = params['channel_ids']
    gain_to_uV = params['gain_to_uV']
    file.close()

    params = si.read_python(str(phy_folder / "params.py"))
    sample_rate = params["sample_rate"]
    rec_data_path = params['dat_path']
    data_type = params['dtype']
    headeroffset = params['offset']
    rec_all = si.read_binary(rec_data_path, sampling_frequency=sample_rate, dtype=data_type, file_offset=headeroffset, \
                            num_channels=len(channel_ids), channel_ids=channel_ids, offset_to_uV=0, gain_to_uV=gain_to_uV, is_filtered=True)

The 'gain_to_uV' in reportinfo.json is 1. And the plotting code is following.

    for unit_id in sorted(sorting.unit_ids, reverse=True):
        unit_sa = sorting_analyzer.select_units(unit_ids=[unit_id])
        wv = unit_sa.get_extension(extension_name="waveforms")
        waveforms = wv.get_data()
        mean_waveform = np.mean(waveforms[:, :, ch], axis=0)
        fig, axes = plt.subplots(nrows=1, ncols=2, width_ratios=[0.7, 0.3])

        max_plot_wf = 100 if waveforms.shape[0] > 100 else waveforms.shape[0]
        for i in range(max_plot_wf):
            axes[0].plot(waveforms[i, :, ch], alpha=0.3, color='grey',linewidth=0.5)
        axes[0].plot(mean_waveform, color='black',linewidth=1)
     ...

In fact, in the units of the sorting result, some are same as the picture in Phy, but some are upside down.

BEGINRX commented 5 days ago
sorting_curated_phy = se.read_phy(phy_folder, exclude_cluster_groups=["noise"])
rec_all = si.read_binary(rec_data_path, sampling_frequency=sample_rate, dtype=data_type, file_offset=headeroffset, \
                            num_channels=len(channel_ids), channel_ids=channel_ids, offset_to_uV=0, gain_to_uV=gain_to_uV, is_filtered=True)
sorting_analyzer = si.create_sorting_analyzer(sorting=sorting_curated_phy, recording=rec_all, sparse=False)
zm711 commented 1 day ago

Sorry is your "binary_file" saved post-filtering + referencing/preprocessing? The SortingAnalyzer performs best with filtered and referenced data. So I want to make sure we are seeing this.

BEGINRX commented 1 day ago

Thank you for your reply. I know it's difficult to find the solution without the data. The binary_file is saved with Phy, so I think the waveforms reconstructed by Phy are same as that I read for plotting. And the data saving process is like this:

    sa = si.create_sorting_analyzer(sorting=sorting, recording=rec_correct)
    export_to_phy(sa, phy_res_path, compute_amplitudes=True, compute_pc_features=True, remove_if_exists=True)

The "rec_correct" is preprocessed using the pipeline "common_reference" -> "bandpass_filter" -> "correct_motion".