xcube-dev / xcube

xcube is a Python package for generating and exploiting data cubes powered by xarray, dask, and zarr.
https://xcube.readthedocs.io/
MIT License
183 stars 19 forks source link

Cannot build levels from SST example dataset #970

Open felixcremer opened 4 months ago

felixcremer commented 4 months ago

I followed the SST example in the docs to generate a datacube and tried to generate levels for this data cube. This fails with a very generic looking cannot determine data type for data resource error.

To Reproduce Steps to reproduce the behavior:

  1. Generate datacube from the examples data following the docs https://xcube.readthedocs.io/en/latest/examples/xcube_gen.html
  2. Run xcube level on the resulting zarr dataset
  3. See error

Expected behavior I expected to get levels folders inside of the .zarr folder. For the .nc files I can generate a .levels folder.

Screenshots

$ xcube level test/data/demo_SST_xcube.zarr/
ERROR: Internal error: Cannot determine data type for  data resource 'test/data/demo_SST_xcube.zarr/'
$ xcube --version
xcube, version 1.5.1

Additional context I am trying to understand how xcube saves the multiscale levels so that I can follow this in the PyramidScheme.jl package so that we can read and write data in Julia that is compatible with xcube. Is there an example xcube data cube with levels available somewhere?

forman commented 4 months ago

Hi Felix,

Sorry, I currently don't have the time to investigate further. But you could, instead of using the xcube level CLI, try the following to create the levels file:

from xcube.core.store import new_data_store

store = new_data_store("file", root="<your-folder>")
ds = store.open_data("<your-dataset>.zarr")
store.write_data(ds, "<your-mldataset>.levels")

For details, please refer to examples/notebooks/datastores/5_multi_level_datasets.ipynb.