bids-standard / bids-specification

Brain Imaging Data Structure (BIDS) Specification
https://bids-specification.readthedocs.io/
Creative Commons Attribution 4.0 International
274 stars 157 forks source link

Add "precomputed" as allowed format for microscopy #1942

Open yarikoptic opened 4 hours ago

yarikoptic commented 4 hours ago

Your idea

Looking at https://bids.neuroimaging.io/bep031 there seems to be no mentioning of "precomputed" file format heavily used by various groups for EM data and supported by a wide range of software (neuroglancer, cloudvolune, tensorstore , webKnossos, etc). It might have been much of a "community standard" but has been established and used for many years with some features which are only coming to Zarr these days (e.g. sharding). Moreover there are (I was told) TBs (if not PBs) of data already in that format so it seems that we might be missing an opportunity to provide support for something which is widely used and cannot be "easily" or "cheaply" converted to another (e.g. Zarr) format.

attn @bids-standard/bep031 - does anyone recall precomputed being discussed/dismissed as an option and why? After all we already support not just Zarr but also other formats (from https://bids-specification.readthedocs.io/en/stable/modality-specific-files/microscopy.html#file-formats):

Microscopy raw data MUST be stored in one of the following formats:

effigies commented 4 hours ago

I suppose https://github.com/google/neuroglancer/blob/3efc904/src/datasource/precomputed/README.md is what you mean by "precomputed"?

yarikoptic commented 3 hours ago

Yes, sorry for the omission