navis-org / pymaid

Python library to interface with CATMAID servers. Fully interoperable with navis.
https://pymaid.readthedocs.io/en/latest/
GNU General Public License v3.0
23 stars 11 forks source link

Fetching image data from N5 sources #238

Closed clbarnes closed 8 months ago

clbarnes commented 1 year ago

I have an implementation which uses the metadata CATMAID stores about N5 stacks to build an xarray.DataArray (wrapping a dask.Array wrapping a zarr.Array...), which users can then query using project coordinates and fetch in parallel. Here, the heavy lifting is done by other libraries so it's only really metadata-wrangling; it can also be extended to other tile source types fairly easily (I'm working on supporting JPEG stacks, where you just need a shim which looks like a zarr.Store).

Would pymaid be the right place for this? It is obviously useful to the same people, but it also brings in some heavy dependencies: zarr, fsspec[http], pydantic>=2, xarray[parallel], imageio (these are currently tucked behind an extra, and the module isn't loaded on first import). Alternatively, I have another package which builds xarray.DataArrays out of chunked stores with different metadata standards (currently OME-NGFF zarr, bigdataviewer n5, n5-viewer, neuroglancer n5), so I could add it there, adding pymaid as a dependency. Much less discoverable for pymaid users, though, even if pointed to in the docs.

schlegelp commented 1 year ago

I think this has a place in pymaid, and not having it imported at top-level and with optional dependencies seems like the right approach.

clbarnes commented 8 months ago

Resolved by https://github.com/navis-org/pymaid/pull/235