Whether sorting data should reside in the same .nwb file as the raw timeseries data

spikodrome / administrative

An administrative area for orchestrating the project

Apache License 2.0

0 stars 0 forks source link

Whether sorting data should reside in the same .nwb file as the raw timeseries data #7

Closed magland closed 4 years ago

magland commented 4 years ago

A .nwb file is meant to encapsulate all aspects of an experiment, and I suppose all data would normally go into the same file. But this is sort of incompatible with the idea of spikeforest where you have many sortings of the same data. I like to keep the raw ephys data separate from the sorting results. Similar for ground truth times/labels. I prefer to have those in a separate file. Any comments about this?

@bendichter @yarikoptic

bendichter commented 4 years ago

Yeah that's fine. Each sorting result cab be it's own file

magland commented 4 years ago

Okay, and how about the ground truth? One could argue that this should go in the same file, but I prefer to keep it separate so it can be treated like any other sorting result. Sound okay @bendichter ?

yarikoptic commented 4 years ago

Keeping in a separate file is also inline with

practices in Allen institute (different users/PIs might have different preferences on preprocessing/sorting)
our YODA principles for "modular components"

bendichter commented 4 years ago

NWB allows you to store all session data in a single file but doesn't require it. For now I think whatever is most convenient is fine. Storing ground truth separate sounds reasonable

magland commented 4 years ago

Great. Closing

yarikoptic commented 4 years ago

Okay, and how about the ground truth?

If there is 100% known ground truth obtained at the same time as the "raw data" -- then I would have kept it along, but keeping it aside in a clearly annotated file would be fine with me as well. Eg. in BIDS, although largely due to crippled nature of the .nii file format, even multi-contrast sequences stored in neighboring files