jpeg encoded zarr/n5 chunks

[d-v-b]

Would you have any interest in adding support to decoding zarr/n5 array chunks as jpegs? Adding jpeg compression to zarr is simple enough (I think we would have to agree on C or F ordering for re-arranging 3D images to big 2D images...) and I think I could get the n5 maintainer on board with the idea.

[jbms]

It would be very easy to support in Neuroglancer since there is already jpeg decoding support used for other formats. I don't see any issue adding it to Neuroglancer, provided that it is upstreamed into the zarr/numcodecs repositories as well, since I want to avoid incompatibility. Not sure if there is a single canonical n5 implementation at this point, but zarr/n5.py might serve as one.

I think for C or F order there is already a reasonably clear answer --- for zarr, use the order specified in the metadata, and for n5, use F order. Presumably you would require data type to be uint8. However, there are other variables relevant to jpeg encoding: which dimensions, if any, to stuff into the width, height, and channel dimensions. Also jpeg has various color space and subsampling options that have a pretty major impact. So your jpeg compression codec may need quite a few parameters.

[d-v-b]

Regarding the planarization of 3D images, how do you do it in the precomputed format?

[jbms]

width dimension corresponds to x, height dimension corresponds to y and z, where consecutive y values correspond to consecutive rows of the image.

The channels of the precomputed format map directly to jpeg channels, and there must be 1 or 3 channels. I've only ever used 1 channel, and in the case of 1 channel the subsampling/color options aren't an issue.

[d-v-b]

I can generate n5 / zarr chunks that would conform to this specification. Because this is a pretty specialized use-case (uint8 only, XYZC only), I doubt it would be in-scope for addition to the core zarr-python library, but zarr makes it very easy to define custom codecs. I'd be happy open-sourcing the codec (potentially we could merge it into the neuroglancer python library).

[jbms]

I see numcodecs has a few codecs but is fairly limited and doesn't have any image codecs, so I imagine you may be right that the authors may not want to include jpeg.

I just want to avoid fragmentation --- by defining a codec for zarr or n5 you will also be claiming a codec id (e.g. "jpeg"), but even if you pick a qualified name like "dvb_jpeg" or "neuroglancer_jpeg", in the future other software may need to interoperate with this format anyway.

I think I would suggest using the codec id of "jpeg" but trying to define the format parameters in a way that avoids problems later, and also at least running it by the n5 and numcodecs authors to hear any objections they may have to claiming the jpeg codec id.

The best way to avoid problems later is to start with the simplest form that you need, and doesn't define any parameters at all.

If you don't need multi-channel jpeg, I'd suggest just supporting grayscale for now. If order == "C", the last dimension goes into the width, and the remaining dimensions go into height. If order == "F", the first dimension goes into width and the remaining dimensions go into height. (For N5, order == "F", but note that zarr/n5.py does a confusing transpose.) Just have a single "quality" parameter (integer from 0 to 100) that is used for encoding (no effect on decoding).

This is simple enough that it is unlikely to create headaches later, and could be extended easily enough to add more parameters.

[d-v-b]

we have tentative java and python implementations of jpeg compression for chunks. The current implementation of n5 in zarr-python only allows a specific set of chunk encodings (of which our jpeg encoding is not a member); I submitted a PR to fix this, but it could take a long time before that gets addressed.

Both implementations only support jpeg and the only parameter is quality (0-100). I haven't added support for "F" ordered arrays but this is easy enough to add.