search

xcube-dev / xcube

xcube is a Python package for generating and exploiting data cubes powered by xarray, dask, and zarr.
https://xcube.readthedocs.io/
MIT License
201 stars 20 forks source link

server fails to return time series for shapes in some cases #995

Closed TejasMorbagal closed 5 months ago

TejasMorbagal commented 5 months ago

Describe the bug server fails to return time series for shapes

[E 2024-05-31 08:15:06,024 tornado.application] Uncaught exception POST /api/timeseries/ds1/B01?aggMethods=mean%2Cstd&startDate=2021-03-05T10%3A35%3A40Z&endDate=2022-10-19T10%3A45%3A38Z (10.0.73.252)
HTTPServerRequest(protocol='http', host='open-sr.api.dev.brockmann-consult.de', method='POST', uri='/api/timeseries/ds1/B01?aggMethods=mean%2Cstd&startDate=2021-03-05T10%3A35%3A40Z&endDate=2022-10-19T10%3A45%3A38Z', version='HTTP/1.1', remote_ip='10.0.73.252')
Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/tornado/web.py", line 1790, in _execute
    result = await result
             ^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/server/webservers/tornado.py", line 339, in post
    await self._call_method("post", *args, **kwargs)
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/server/webservers/tornado.py", line 354, in _call_method
    await method(*args, **kwargs)
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/routes.py", line 82, in post
    result = await self.ctx.run_in_executor(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/controllers.py", line 101, in get_time_series
    results = _get_time_series_for_geometries(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/controllers.py", line 137, in _get_time_series_for_geometries
    time_series = _get_time_series_for_geometry(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/controllers.py", line 176, in _get_time_series_for_geometry
    time_series_ds = timeseries.get_time_series(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/timeseries.py", line 138, in get_time_series
    dataset = mask_dataset_by_geometry(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/geom.py", line 337, in mask_dataset_by_geometry
    dataset = _clip_dataset_by_geometry(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/geom.py", line 509, in _clip_dataset_by_geometry
    update_dataset_spatial_attrs(dataset_subset, update_existing=True, in_place=True)
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/update.py", line 78, in update_dataset_spatial_attrs
    gm = GridMapping.from_dataset(dataset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/gridmapping/base.py", line 803, in from_dataset
    return new_grid_mapping_from_dataset(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/gridmapping/dataset.py", line 89, in new_grid_mapping_from_dataset
    raise ValueError("cannot find any grid mapping in dataset")
ValueError: cannot find any grid mapping in dataset
[E 2024-05-31 08:15:06,024 tornado.application] Uncaught exception POST /api/timeseries/ds1/B01?aggMethods=mean%2Cstd&startDate=2021-03-05T10%3A35%3A40Z&endDate=2022-10-19T10%3A45%3A38Z (10.0.73.252)
HTTPServerRequest(protocol='http', host='open-sr.api.dev.brockmann-consult.de', method='POST', uri='/api/timeseries/ds1/B01?aggMethods=mean%2Cstd&startDate=2021-03-05T10%3A35%3A40Z&endDate=2022-10-19T10%3A45%3A38Z', version='HTTP/1.1', remote_ip='10.0.73.252')
Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/tornado/web.py", line 1790, in _execute
    result = await result
             ^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/server/webservers/tornado.py", line 339, in post
    await self._call_method("post", *args, **kwargs)
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/server/webservers/tornado.py", line 354, in _call_method
    await method(*args, **kwargs)
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/routes.py", line 82, in post
    result = await self.ctx.run_in_executor(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/controllers.py", line 101, in get_time_series
    results = _get_time_series_for_geometries(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/controllers.py", line 137, in _get_time_series_for_geometries
    time_series = _get_time_series_for_geometry(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/webapi/timeseries/controllers.py", line 176, in _get_time_series_for_geometry
    time_series_ds = timeseries.get_time_series(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/timeseries.py", line 138, in get_time_series
    dataset = mask_dataset_by_geometry(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/geom.py", line 337, in mask_dataset_by_geometry
    dataset = _clip_dataset_by_geometry(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/geom.py", line 509, in _clip_dataset_by_geometry
    update_dataset_spatial_attrs(dataset_subset, update_existing=True, in_place=True)
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/update.py", line 78, in update_dataset_spatial_attrs
    gm = GridMapping.from_dataset(dataset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/gridmapping/base.py", line 803, in from_dataset
    return new_grid_mapping_from_dataset(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/xcube-1.6.0.dev2-py3.12.egg/xcube/core/gridmapping/dataset.py", line 89, in new_grid_mapping_from_dataset
    raise ValueError("cannot find any grid mapping in dataset")
ValueError: cannot find any grid mapping in dataset
forman commented 5 months ago

image

forman commented 5 months ago

image

forman commented 5 months ago

Explanation: For a given shape, the dataset is masked and then cropped to shape bounds, finally its global CF coverage attributes are updated. The latter step fails. It seems the dataset changes such that GridMapping.from_dataset(dataset) fails.

forman commented 5 months ago

Subsetting variables is the root cause. Since the server picks a single variable, the subset dataset only comprises that variable and no longer crs.