lexical error when reading SpatialData example OME-NGFFs

[lgeistlinger]

Hi @keller-mark,

@lee-t and I were just playing around with pizarr and tried to use it to load some of the SpatialData example datasets from here:

https://spatialdata.scverse.org/en/latest/tutorials/notebooks/datasets/README.html

However, we encountered the following problems:

> library(pizzarr)
> root <- "https://s3.embl.de/spatialdata/spatialdata-sandbox/toy.zarr"
> store <- HttpStore$new(root)
Error in loadNamespace(x) : there is no package called ‘crul’

It seems crul is in Suggests but it would rather need to go into Imports. Or alternatively keep it in Suggests if it is not considered essential functionality for the package and throw an error with a prompt for the user to install the crul package. But here it seems to be an essential part as it is part of creating a new HttpStore object.

Ok, proceeding after installing crul:

> store <- HttpStore$new(root)
> g <- zarr_open_group(store)
> attrs <- g$get_attrs()$to_list()
Error: lexical error: invalid char in json text.
                                       <?xml version="1.0" encoding="U
                     (right here) ------^

The same happens when using another dataset as root eg, when using

root <- "https://s3.embl.de/spatialdata/spatialdata-sandbox/cosmx_io.zarr"

[keller-mark]

Thanks for the issue. Pizzarr should definitely produce a better error in this case, but there are ultimately no attributes at that location at the root of the store.

This is arising because SpatialData does not store a .zattrs file in the root of the Zarr store, so the result of the request to https://s3.embl.de/spatialdata/spatialdata-sandbox/cosmx_io.zarr/.zattrs (which is being made upon calling g$get_attrs() on the root) is an S3 NoSuchKey response

<?xml version='1.0' encoding='UTF-8'?><Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Details>No such object: spatialdata/spatialdata-sandbox/cosmx_io.zarr/.zattrs</Details></Error>

https://s3.embl.de/spatialdata/spatialdata-sandbox/cosmx_io.zarr/.zattrs

[keller-mark]

There is a zattrs file at https://s3.embl.de/spatialdata/spatialdata-sandbox/toy.zarr/images/image/.zattrs

root <- "https://s3.embl.de/spatialdata/spatialdata-sandbox/toy.zarr"
store <- HttpStore$new(root)
g <- zarr_open_group(store)
attrs <- g$get_item("/images/image")$get_attrs()$to_list()

[lgeistlinger]

Thanks Mark. Yes it seems that only the image part of SpatialData is stored as OME-NGFF within the overall SpatialData zarr store.

[keller-mark]

Yes, there will also be zattrs files at other places within a SpatialData object such as within the tables part, which can be seen by downloading the Zarr zip file and exploring locally

[lgeistlinger]

Great, closing, this does the trick. Thanks!