Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)
Apache License 2.0
26 stars 4 forks source link

export_workspace: skip writing to local storage #883

Open jdries opened 5 days ago

jdries commented 5 days ago

When doing save_result => export_workspace, the data is stored once in default location and once in workspace. This is problematic for huge processing jobs, because we will have to pay for the storage.

bossie commented 13 hours ago

@jdries What are the implications from the user's point of view? Does this mean that the STAC Collection of batch job results has no items then (also because OpenEO won't be able to serve the assets anymore)?

jdries commented 9 hours ago

or can we make those items point to the location where we exported the results to? If complex, the main thing to achieve here is simply that the file is gone, potentially triggered by a feature flag or job option if the impact is too unpredictable.