search

Open-EO / openeo-gfmap

Generic framework for EO mapping applications building on openEO
Apache License 2.0
3 stars 0 forks source link

Don't overwrite existing STAC collection when doing a new extraction #94

Closed VincentVerelst closed 1 month ago

VincentVerelst commented 1 month ago

Currently, when an extraction is done, the job_manager will always create a new STAC collection from scratch. This is inconvenient, if you want to do extra extractions on top of previous work.

Proposed solution: allow an existing STAC collection to be given as an input to the job_manager. The job_manager will then add the new extractions as items to the existing collection. In case no STAC collection is provided to the job_manager, it will create one from scratch.

VincentVerelst commented 1 month ago

Currently https://github.com/Open-EO/openeo-python-client/issues/558 is blocking this. To be discussed if this needs to be adressed in GFMap or in the Python Client.

GriffinBabe commented 1 month ago

Another way could be to search automatically for a stac catalogue in the output path.

If there is a stac catalogue, then make sure that the AssetDefinitions are the same (so we don't extend a STAC catalogue with another containing different type of extractions). If they are same, we load the stac catalogue in the manager and extend it with new item/assets.

If no stac catalogue, start from scratch as we do now.

I linked this issue to a more generic issue about Job Manager and Extraction processes in general to be more robust