Error running ap_registration.ipynb from Jupyter Notebook

[scg63]

We are considering running DREDge for motion correction prior to running KS4, so I've installed DREDge (I am using Conda instead of Mamba, already using Conda for running kilosort and phy) and planned to run it with the example datasets mentioned (https://figshare.com/articles/dataset/_Imposed_motion_datasets_from_Steinmetz_et_al_Science_2021/14024495). I've downloaded the dataset1 from Steinmetz et al 2021 to use in ap_registration.ipynb in Jupyter Notebook as an example run, and replaced the path with my own in cell 4. I had received an error when running this chunk stating I needed to install ipywidgets, so I followed the instructions mentioned in the error message (installed ipywidgets in my dredge environment). After running the script again, I've received the following error message: KeyError: "key (0, 'imec0.ap') is already in the signals_info_dict"

I've attached an image of the traceback. I will keep looking into this, but I'd be greatly appreciative if anyone has any insight to what the error is referring to. I'm hoping it is a simple fix, or that I'm missing something since I have insufficient knowledge of how Jupyter Notebook/Python works! I work primarily with Matlab. Please see the attached images with the traceback error.

[JoeZiminski]

Hi @scg63, would you mind showing the folder structure of the data you are trying to load (e.g. with this tool[https://tree.nathanfriend.io/])?

I am not familiar with that error but I can see it has been reported before and it might be related to what folder read_spikeglx is being pointed too, this can be a little finicky. It might be worth trying to pass the folder a level up / down.

[JoeZiminski]

(also if in doubt when using a notebook, try restarting the kernel to run the script 'fresh')

[scg63]

Thank you @JoeZiminski for responding. The folder structure is: . └── Downloads/ └── dataset1/ ├── p1_g0_10.imec0.ap.meta ├── p1_g0_t0.imec0.ap.bin ├── p2_g0_t0.imec0.ap.bin ├── p2_g0_t0.imec0.ap.meta ├── NP2_kilosortChanMap.mat ├── neuropixPhase3B1_kilosortChanMap_all.mat ├── manip.timestamps_p2.npy ├── manip.timestamps_p1.npy └── manip.positions.npy

I spoke with @cwindolf who mentioned it could be due to the structuring of the folder (the .meta and .bin folders for the two parts are not separated) and it seems the function called could not distinguish which file to use. I reorganized the folder with the example data provided and the issue was resolved. Thank you for your response!

[cwindolf]

I wonder if the error message could be a bit more informative in this case?

[JoeZiminski]

Yes @cwindolf I agree I often have trouble understanding what is going wrong when trying to load spikeglx, for example if the folder cannot be found the error is quite cryptic IIRC