xcube-dev / xcube

xcube is a Python package for generating and exploiting data cubes powered by xarray, dask, and zarr.
https://xcube.readthedocs.io/
MIT License
201 stars 20 forks source link

xcube serve to offer same enhancements for DataStores as for Datasets #638

Closed AliceBalfanz closed 2 years ago

AliceBalfanz commented 2 years ago

Is your feature request related to a problem? Please describe. When configuring a xcube server for specific datasets, I can specify e.g. an augmentation added to the dataset or using a specific script for resampling in time. These enhancements happen on the fly and allow us to customise output in the viewer further. This functionality is not given for Datasets served via DataStores.

Describe the solution you'd like I would like to be able to specify e.g. augmentation, which are performed on all datasets served in the specified DataStore identifier. Same for resampling in time.

Describe alternatives you've considered The only alternative is, that the datacubes are already exactly in the form as they are needed for the customer. This is not very flexible, because when changes are requested it might lead to regenerating many cubes.

Additional context Configuration example of Datasets in a xcube server config

Datasets:
  - Identifier: cyanoalert-lm
    Title: "Lake Mälaren"
    FileSystem: "s3"
    Endpoint: "https://s3.eu-central-1.amazonaws.com"
    Anyonymous: false
    Path: "path/to/Sweden-v3.zarr"
    Style: default
    TimeSeriesDataset: cyanoalert-lm_time_chunk
    PlaceGroups:
      - PlaceGroupRef: Sweden_Lake_Malaren_bodies
    Augmentation:
      Path: "/xcube/resources/dev/api/augmentation/get_custom_variables_malaren.py"
      Function: "get_custom_vars"
      InputParameters:
        vars_for_masking: ["floating_vegetation", "immersed_cyanobacteria"]
        details_for_masking: "/xcube/resources/details_for_masking/details_for_masking_malaren.yml"
        geometry: "/xcube/resources/places/Sweden_Lake_Malaren_waterbodies.geojson"

Related issue: https://github.com/dcs4cop/xcube/issues/577

AliceBalfanz commented 2 years ago

Tested again with xcube master version, and it works! Closing