search

GeoscienceAustralia / dea-orchestration

4 stars 1 forks source link

Stac - valid data and AWS event filter prefix update and tests #100

Closed ashoka1234 closed 5 years ago

ashoka1234 commented 5 years ago

Request for this pull request

Geometry Compute Update and Terraform cannot have overlapping prefixes for event filters. Added pytests.

Proposed solution

  1. Not all the database products specify valid_data. Use geo_ref_points to compute geometry when valid_data is absent.
  2. Remove filter_prefix from aws_s3_bucket_notification in terraform script. Rely on lambda function in stac.py for validation of yaml filenames to ensure the notified file indeed corresponds to a dataset of a product.
  3. Pytest fixtures are added for testing STAC catalog updates and the function of lambda function that creates the STAC item catalogs. However travis needs credentials for this to work for automation.
omad commented 5 years ago

Ashoka, can you please clean up the commits in this PR before I look at it.

ashoka1234 commented 5 years ago

Done! If you need more squashing please let me know. This PR seems to have picked commits from my previous PR on this repo. Should have started new branch for this.

omad commented 5 years ago

Thanks, that's much tidier for the commits, but you'll need to rebase as well so that it can merge cleanly to master. At the moment there's at least six files with merge errors.