search

Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)
Apache License 2.0
26 stars 5 forks source link

`load_result` failure #127

Open soxofaan opened 2 years ago

soxofaan commented 2 years ago

originally reported here https://discuss.eodc.eu/t/caching-openeo-results/197/22 and related to #126

synchronous download of a process graph with load_result results in 

OpenEoApiError: [500] unknown: / by zero
soxofaan commented 2 years ago

I could reproduce this with a NetCDF download attempt and found this in logs:

22/02/24 15:58:22 INFO netcdf.NetCDFRDDWriter$: Writing NetCDF from rdd with : 0 elements and 0 partitions.
22/02/24 15:58:22 INFO netcdf.NetCDFRDDWriter$: Writing NetCDF from rdd with : 0 elements and 0 partitions.
....
  File ".../site-packages/openeo_driver/save_result.py", line 81, in save_result
    return self.cube.save_result(filename=filename, format=self.format, format_options=self.options)
  File ".../site-packages/openeogeotrellis/geopysparkdatacube.py", line 1356, in save_result
    result = self.write_assets(filename, format, format_options)
  File ".../site-packages/openeogeotrellis/geopysparkdatacube.py", line 1599, in write_assets
    asset_paths = self._get_jvm().org.openeo.geotrellis.netcdf.NetCDFRDDWriter.saveSingleNetCDF(max_level.srdd.rdd(),
  File "../lib/py4j-0.10.9.2-src.zip/py4j/java_gateway.py", line 1309, in __call__
    return_value = get_return_value(
  File ".../lib/py4j-0.10.9.2-src.zip/py4j/protocol.py", line 326, in get_return_value
    raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling z:org.openeo.geotrellis.netcdf.NetCDFRDDWriter.saveSingleNetCDF.
: java.lang.ArithmeticException: / by zero
    at org.openeo.geotrellis.netcdf.NetCDFRDDWriter$.saveSingleNetCDF(NetCDFRDDWriter.scala:144)
    at org.openeo.geotrellis.netcdf.NetCDFRDDWriter.saveSingleNetCDF(NetCDFRDDWriter.scala)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
    at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
    at java.base/java.lang.Thread.run(Thread.java:834)
soxofaan commented 2 years ago

minimal example to reproduce: https://gist.github.com/soxofaan/b69c54acc51a36f47b9284da2ca04e8e

soxofaan commented 2 years ago

(internal ref: EP-4165)

jdries commented 2 years ago

Cause is quite clearly an empty data cube: https://github.com/Open-EO/openeo-geotrellis-extensions/blob/feature/spark3/openeo-geotrellis/src/main/scala/org/openeo/geotrellis/netcdf/NetCDFRDDWriter.scala#L144

So we clearly need a better error, or an empty netcdf file.