Open-EO / openeo-spring-driver

openEO driver build on spring using WCPS and ODC storage access
Apache License 2.0
0 stars 0 forks source link

Refactor BatchJobResult class to handle STAC Assets and Collections #96

Closed clausmichele closed 1 month ago

clausmichele commented 4 months ago

So far, the implementation of a batch job result supposed to be always a STAC Asset (type: "Feature").

The openEO API allows to return batch job results also as a STAC Collection, see here: https://api.openeo.org/#tag/Batch-Jobs/operation/list-results

We have to be able to handle both scenarios.

This sample process graph can be used to quickly test a batch job with a geoTIFF result which is returned as STAC Collection:

{
  "process_graph": {
    "save1": {
      "result": true,
      "process_id": "save_result",
      "arguments": {
        "data": {
          "from_node": "load1"
        },
        "format": "geotiff"
      }
    },
    "load1": {
      "process_id": "load_collection",
      "arguments": {
        "temporal_extent": [
          "2023-01-08T00:00:00Z",
          "2023-01-08T23:59:59Z"
        ],
        "spatial_extent": {
          "east": 11.4,
          "south": 46.45,
          "north": 46.47,
          "west": 11.38
        },
        "id": "s2_l2a",
        "bands": [
          "blue",
          "green",
          "red"
        ]
      }
    }
  }
}
pierocampa commented 3 months ago

Which branch should I be starting to work on, the feature/stac_results, right?

clausmichele commented 3 months ago

@pierocampa yes exactly.

pierocampa commented 3 months ago

So the "new scenario" would be a STAC backend giving back a response? Is the "engine" field of the job giving a hint that this is a STAC backend?

Because now the handling of these response is inside this branching:

if (job.getEngine()==EngineTypes.ODC_DASK)
    ...
clausmichele commented 3 months ago

The new EngineTypes that we discussed about in the last meeting could be the one for the OGC Application Package runner, so not related to this. In this case we just need to be able to return a STAC Collection instead of a STAC Item, which is currently limited by the implementation of the class BatchJobResult.

pierocampa commented 3 months ago

Update: still working on it. Unit tests are there, but there are still issues with Hibernate/DAOs. 75b79e1

pierocampa commented 3 months ago

@clausmichele, I just pushed the last commit (8840559): I am currently unable to make a quick smoke test on the proper behaviour of the server now with both STAC Features and Collections options as job results.

Code compiles, and all unit tests pass, that's what I know for the moment. Let me know on Monday if possible if you are able to verify the submission of jobs. thx

pierocampa commented 1 month ago

Latest update ready for review or testing (8b1ea1c4).

The patch adds the STAC Collection as option for a result coming from the ODC backend in addition to the "legacy" custom result type which is a JSON pointing to the output file, from which a STAC Feature is manually crafted.

The pre-existing (weak) switch condition is based on parsing a stac_version element in the JSON repsonse from the ODC server (see JobScheduler::executeODC(...)).

Integration tests added to the TestBatchJobResultCollection suite, based on the oracle in the stac.collection.200.json (and also live-tested against running ODC backend which returns a collection like the one in stac.collection.sl2.json)