search

Open-EO / openeo-geotrellis-extensions

Java/Scala extensions for Geotrellis, for use with OpenEO GeoPySpark backend.
Apache License 2.0
5 stars 3 forks source link

filter_temporal: bug when applied #208

Closed jdries closed 10 months ago

jdries commented 10 months ago

JOBID : openEO batch_Cropclass_TrainingFeat_2021_BLZ_ORWLK_POINT_110_OTHER_CROPS_terrascope_j-3a65a12ce57d46a9aee8cb9499c4c247_user 5e15639083f8407aaf106d87fd5bc26738619baf2bdc48bece79b1225ffbcaee@egi.eu

 File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1472, in <dictcomp>
    args = {name: convert_node(expr, env=env) for (name, expr) in sorted(args.items())}
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 380, in convert_node
    return convert_node(processGraph['node'], env=env)
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 368, in convert_node
    process_result = apply_process(process_id=process_id, args=processGraph.get('arguments', {}),
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1504, in apply_process
    return process_function(args=ProcessArgs(args, process_id=process_id), env=env)
  File "/opt/venv/lib64/python3.8/site-packages/openeo_driver/ProcessGraphDeserializer.py", line 1140, in filter_temporal
    return cube.filter_temporal(start=extent[0], end=extent[1])
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/geopysparkdatacube.py", line 204, in filter_temporal
    return self.apply_to_levels(
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/geopysparkdatacube.py", line 158, in apply_to_levels
    pyramid = Pyramid({k: func(l) for k, l in self.pyramid.levels.items()})
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/geopysparkdatacube.py", line 158, in <dictcomp>
    pyramid = Pyramid({k: func(l) for k, l in self.pyramid.levels.items()})
  File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/geopysparkdatacube.py", line 205, in <lambda>
    lambda rdd: rdd.filter_by_times([pd.to_datetime(start), pd.to_datetime(end)]),
  File "/opt/venv/lib64/python3.8/site-packages/geopyspark/geotrellis/layer.py", line 2213, in filter_by_times
    return TiledRasterLayer(self.layer_type, result)
  File "/opt/venv/lib64/python3.8/site-packages/geopyspark/geotrellis/layer.py", line 1081, in __init__
    self.layer_metadata = Metadata.from_dict(json.loads(self.srdd.layerMetadata()))
  File "/opt/venv/lib64/python3.8/site-packages/geopyspark/geotrellis/__init__.py", line 799, in from_dict
    if len(bounds_dict['minKey']) == 2:
KeyError: 'minKey'
jdries commented 10 months ago

happens when applying temporal filter two times, proposed solution would be to avoid using geopyspark

jdries commented 10 months ago

got fixed in aggregate_temporal, special case for sparse partitioner