Test out zarr back-end for NWB files

LorenFrankLab / spyglass

Neuroscience data analysis framework for reproducible research built by Loren Frank Lab at UCSF

https://lorenfranklab.github.io/spyglass/

MIT License

94 stars 43 forks source link

Test out zarr back-end for NWB files #1134

Open lfrank opened 1 month ago

lfrank commented 1 month ago

My understanding is that zarr is strongly preferred for cloud-based computing, and I'm wondering if we should try out the zarr back-end. Ideally this would also be something we might move to for initial NWB file creation.

This is not our highest priority, but could be worth considering.

rly commented 1 month ago

Yes, Zarr is strongly preferred for cloud-based computing. The Allen Institute for Neural Dynamics is using NWB in Zarr already. But I want to note that the Zarr backend has some rough edges, especially around changing the format (exporting) of an NWB HDF5 file to Zarr and back. We would love to have more users to test out the zarr backend, though!