Properly pass config to batch jobs

Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)

Apache License 2.0

26 stars 4 forks source link

Properly pass config to batch jobs #464

Closed soxofaan closed 1 year ago

soxofaan commented 1 year ago

while working on #455 I noticed that batch jobs do not get the active GpsBackendConfig (because OPENEO_BACKEND_CONFIG is not properly propagated), but fall back on the default config (https://github.com/Open-EO/openeo-geopyspark-driver/blob/master/openeogeotrellis/config/default.py)

jdries commented 1 year ago

CDSE still has some batch jobs where config does not seem to be passed on for some reason.

soxofaan commented 1 year ago

Eliminated the last straw: the Terrascope job tracker nifi runner was running with an outdated image apparently. Updated it to latest prod image and now the last indication of "wrongly" loading config is not being added anymore in the logs