UDP support (`/process_graphs`)

Open-EO / openeo-aggregator

openEO driver that combines multiple other drivers

https://open-eo.github.io/openeo-aggregator/

Apache License 2.0

1 stars 1 forks source link

UDP support (`/process_graphs`) #90

Closed soxofaan closed 1 year ago

soxofaan commented 1 year ago

Add support for UDPs (GET /process_graphs and related).

Questions:

where to store? at level of aggregator, or "highest prio" upstream back-end, or all backends?
what to list? only UDPs defined at aggregator or combine multiple upstream back-ends?
how to handle job with UDP usage if UDPs are stored only at level of aggregator? Automatically store UDP in upstream back-end first, or rewrite "UDP calls" in original process graph with actual UDP implementation?

m-mohr commented 1 year ago

The details have to be discussed, but the most important thing is that files and user-defined processes must be accessible by all back-ends in the federation because otherwise, users can't re-use them properly in other processes. User shouldn't need to care about where something is available and where it's not available.

Patrick1G commented 1 year ago

@m-mohr @soxofaan @jdries UPD support by the aggregator is important for the platform part!
Could we add a task as part of SAP09 Operationalization on this?

soxofaan commented 1 year ago

We just added very basic UDP support to the aggregator (just on dev instance openeocloud-dev.vito.be at the moment): UDP interactions are just proxied directly to VITO back-end: creating, getting, listing UDPs on the aggregator goes directly to VITO UDP storage.

This is not ideal because there is no federation aspect to this, but at least the basics are working for majority of processing requests (which go to VITO anyway).

We just decided this was an easy first step to have basic functionality in place

soxofaan commented 1 year ago

basic implementation is available, will close this ticket.

Follow-up is for https://github.com/openEOPlatform/architecture-docs/issues/368