xcube-dev / xcube

xcube is a Python package for generating and exploiting data cubes powered by xarray, dask, and zarr.
https://xcube.readthedocs.io/
MIT License
201 stars 20 forks source link

Assignment of time-optimized cubes via (configurable) patterns or templates within xcube serve DataStore configuration #577

Open AliceBalfanz opened 2 years ago

AliceBalfanz commented 2 years ago

Is your feature request related to a problem? Please describe. I would like to configure corresponding time optimized cubes via the DataStore configuration of xcube serve.

Current Situation:

Many of our datacubes have a corresponding time optimized cubes, which is used e.g. for time series display in xcube viewer. The time optimized cubes are not visible the datasets of xcube viewer but if they are configured in the xcube server they will be used for the time series generation. The configuration is like the one below:

Datasets:
  - Identifier: l2c-lm
    Title: "Lake Mälaren"
    FileSystem: "obs"
    Endpoint: "https://s3.eu-central-1.amazonaws.com"
    Anyonymous: false
    Path: "bucket/ws-cube-v3.zarr"
    Style: default
    TimeSeriesDataset: l2c-lm_time_chunk

  - Identifier: l2c-lm_time_chunk
    Title: "Lake Mälaren Time Chunked"
    Format: zarr
    Endpoint: "https://s3.eu-central-1.amazonaws.com"
    Anyonymous: false
    Path: "bucket/ws-cube-v3-t_opt.zarr"
    FileSystem: "obs"
    Hidden: true
    Style: default

When using DataStores the configuration would look e.g. like the one below:

DataStores:
  - Identifier: Sweden
    StoreId: s3
    StoreParams:
      root: bucket/Sweden
      max_depth: 1
      storage_options:
        anon: false
    Datasets:
      - Identifier: "ws-*-v3.zarr"
        Style: "default"

It would be great, if xcube serve would discover if there are cubes which are time optimized and then automatically use them for time-series visualisation.

This could be achieved e.g. by agreeing on a certain prefix or suffix in the cubes naming. For example, we have ws-cube-v3.zarr (spatially optimised and visible in xcube viewer) and corresponding ts_ws-cube-v3.zarr

This way it would be possible to match the correct cube pairs.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

forman commented 2 years ago

Note that the new DataStores configuration does not prevent using time-optimized cubes per se. Therefore, the title of this issue is misleading. I believe, what you request is an assignment of time-optimized cubes via (configurable) patterns or templates.