haniffalab / webatlas-pipeline

A data pipeline built in Nextflow to process spatial and single-cell experiment data for visualisation in WebAtlas
MIT License
46 stars 10 forks source link

Multiple resolution sets found. Use 'variable' or 'group' to filter #123

Closed davehorsfall closed 1 year ago

davehorsfall commented 1 year ago

Writing to SpatialData fails when trying to processes the Visium CytAssist sample. Error message shown below.

ERROR ~ Error executing process > 'Full_pipeline:Output_to_spatialdata:write_spatialdata ([visium, breast-cancer])'

Caused by:
  Process `Full_pipeline:Output_to_spatialdata:write_spatialdata ([visium, breast-cancer])` terminated with an error exit status (1)

Command executed:

  write_spatialdata.py         --stem visium-breast-cancer         --anndata_path visium-breast-cancer-anndata.zarr         --raw_img_path tissue_image.tif         --label_img_path visium-breast-cancer-label.tif

Command exit status:
  1

Command output:
  (empty)

Command error:
  Traceback (most recent call last):
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/bin/write_spatialdata.py", line 95, in <module>
      fire.Fire(write_spatialdata)
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/work/conda/webatlas-994912254df9e25b595e2b9871713f0f/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
      component_trace = _Fire(component, args, parsed_flag_args, context, name)
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/work/conda/webatlas-994912254df9e25b595e2b9871713f0f/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
      component, remaining_args = _CallAndUpdateTrace(
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/work/conda/webatlas-994912254df9e25b595e2b9871713f0f/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
      component = fn(*varargs, **kwargs)
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/bin/write_spatialdata.py", line 81, in write_spatialdata
      sdata.add_image("raw", read_image(raw_img))
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/bin/write_spatialdata.py", line 22, in read_image
      imarray = xr.open_dataarray(path, engine="rasterio", mask_and_scale=False)
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/work/conda/webatlas-994912254df9e25b595e2b9871713f0f/lib/python3.10/site-packages/xarray/backends/api.py", line 750, in open_dataarray
      dataset = open_dataset(
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/work/conda/webatlas-994912254df9e25b595e2b9871713f0f/lib/python3.10/site-packages/xarray/backends/api.py", line 573, in open_dataset
      backend_ds = backend.open_dataset(
    File "/home/ndh122/Projects/HaniffaLab/webatlas-pipeline/work/conda/webatlas-994912254df9e25b595e2b9871713f0f/lib/python3.10/site-packages/rioxarray/xarray_plugin.py", line 76, in open_dataset
      raise RioXarrayError(
  rioxarray.exceptions.RioXarrayError: Multiple resolution sets found. Use 'variable' or 'group' to filter.

Work dir:
  /home/ndh122/Projects/HaniffaLab/webatlas-pipeline/work/32/3d9bfaf1bee2dce4f9c1d8911e8400

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
dannda commented 1 year ago

Haven't really found a solution using the suggested variable or group filtering. Can't find what's an appropriate input. In 92f4c1a changed the image loading to dask_image.imread though it doesn't load band metadata as xarray.open_dataarray does with rioxarray. Instead getting image axes with tifffile which might not generalize very well as it's hard to tell whether dims/axes named like "sample" or "sequence" (found in Visium and Xenium sample datasets) are equivalent to a channel or z. Currently setting them as channels. But other axes names might come up from other formats.

dannda commented 1 year ago

Fixed in https://github.com/haniffalab/webatlas-pipeline/commit/92f4c1aec5ffe02da67eb107114407fa020b27fb