phy curation issue, files saved incorrectly

[mariapfj]

We've increasinlgy noticed the same issues we've had with expipe, where channel_group is spontaneously lost during or after curation, causing downstream scripts to err. In one rec (\aspasia.ad.fp.educloud.no\ec109\maria\mec-social-nwb\actions\148-201021-2) I've tried phy_restore, re-registered and spikesorted the rec, and keeping an eye on channel_group during curation to avoid information getting lost, but when I went back to inspect a unit, most (if not all) unsorted units were missing channel_group, and several units previously set as noise were now unsorted (see example image). I have no clue what could be happening, and we have not been able to restore channel_groups in the instances where only a few key units are missing them.

this unit was set as noise but changed to unosrted (with the appropriate information being kept in cluster_info.tsv):

this unit lost channel_group, among other info, and therefore cannot be saved:

I realise this is probably a very convoluted issue but I wanted to inform you and hear if you have any inclination as to what's happening?

[alejoe91]

Let me try to patch this one. There is a function that re-estimates the channel groups when applying curation here, but it's currently run only if spike sorting was not run on groups. I push a fix when this is ran every time there are some missing clusters. Can you send me the phy folder you shared the screenshots for in a zip file?

Note that the groups will be available in the NWB file, but not in Phy since Phy doesn't know how to propagate custom columns after merges/splits

[mariapfj]

Emailed it to you, let me know if you need anything else. Regarding the last point, it's still technically available, and could in theory be extracted from the "peak channel", but without the channel_group we're unable to save the curated outcome to nwb :)