Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)
Apache License 2.0
26 stars 5 forks source link

test export_workspace + asset_per_band combination #905

Closed JeroenVerstraelen closed 1 month ago

JeroenVerstraelen commented 1 month ago

Edit batch job options from existing integrationtest

VictorVerhaert commented 1 month ago

example job that should be able to work: j-241011225b584a8ca14e374da7a9afe4

JeroenVerstraelen commented 1 month ago

In cdse integrationtests, @EmileSonneveld is working on asset_per_band.

soxofaan commented 1 month ago

Edit batch job options from existing integrationtest

@JeroenVerstraelen which existing integration test? I can't find any export_workspace or asset_per_band related test in neither:

VictorVerhaert commented 1 month ago

For this issue we will probably have to create a bucket for integration tests to avoid using (and leaking) workspace names of project specific buckets

EmileSonneveld commented 1 month ago

This integration test would be in a private repo, so not risk of leaking workspace names

soxofaan commented 1 month ago

can't we have the test logic public, but we manage the secrets in the Vault? We already have plenty of tests that work that way

JeroenVerstraelen commented 1 month ago

Requires a bucket for integration tests and a workspace. Alternative: moto library has throwaway s3 test service (scala requires swift url, you'll have to point this to the boto3 service via env variable).

soxofaan commented 1 month ago

Alternative: moto library has throwaway s3 test service

FYI: https://docs.getmoto.org/en/latest/docs/server_mode.html

For example I used that in the tests for a pytest plugin for the APEx benchmarks https://github.com/ESA-APEx/apex_algorithms/blob/7142e499a0ac8b261d6d941274965058e4ac0fda/qa/unittests/tests/pytest/test_pytest_upload_assets.py#L35-L39 using these fixtures https://github.com/ESA-APEx/apex_algorithms/blob/7142e499a0ac8b261d6d941274965058e4ac0fda/qa/unittests/tests/pytest/conftest.py