search

magland / dendro

4 stars 2 forks source link

Extend postprocessing #2

Closed alejoe91 closed 3 months ago

alejoe91 commented 3 months ago

@magland can you try this out?

alejoe91 commented 3 months ago

The " is by running black :P

magland commented 3 months ago

@alejoe91 thanks for this, it's mostly working!

This is already merged, but we can continue the discussion here.

I had to disable a few things because they were giving me errors and I wanted to get it to run through the processing before addressing those.

Here's an example output table! https://neurosift.app/?p=/nwb&url=https://tempory.net/f/pairio/f/hello_world_service/hello_neurosift/spike_sorting_post_processing/mLw5jQ2rTiR8tEbil08m/output/post.nwb.lindi.tar&st=lindi&dandisetId=000409&dandisetVersion=draft&tab=neurodata-item:/processing/ecephys/units_kilosort4|Units

The autocorrelograms are working too! https://neurosift.app/?p=/nwb&url=https://tempory.net/f/pairio/f/hello_world_service/hello_neurosift/spike_sorting_post_processing/mLw5jQ2rTiR8tEbil08m/output/post.nwb.lindi.tar&st=lindi&dandisetId=000409&dandisetVersion=draft&tab=view:Autocorrelograms|/processing/ecephys/units_kilosort4

In this output NWB the original units are in the usual /units location whereas the new units are at /processing/ecephys/units_kilosort4

I think estimated_x, estimated_y, estimated_z should just be x, y, z. Takes up a lot of space and it's obvious that it's estimated, right? Once we decide on this, I can make a neurosift plugin for it.

There was a problem with creating the peak_channel hdf5 dataset so I disabled that, but I think I can fix it.

I also had to disable computing of waveforms. Here's the code. I don't have the exact message because I deleted the job, but it was something about "mean" not being the correct operator. Alessio do you recognize that? If not I can regenerate the error.

# waveform mean and sd
templates_ext = analyzer.get_extension("templates")
template_means = templates_ext.get_templates(operator="mean")
templates_sd = templates_ext.get_templates(operator="std")
colnames.append("waveform_mean")
units.create_dataset(
    "waveform_mean", data=template_means, dtype=template_means.dtype
)
colnames.append("waveform_sd")
units.create_dataset(
    "waveform_sd", data=templates_sd, dtype=templates_sd.dtype
)

@bendichter

magland commented 3 months ago

Oh I just realized a bunch of the metrics are not showing up in the table. Will investigate.

alejoe91 commented 3 months ago

@magland my mistake. Here's a follow up PR with fixed https://github.com/magland/pairio/pull/3