feat: extract dataset publish

NASA-IMPACT / veda-data-airflow

Airflow implementation of ingest pipeline for VEDA STAC data

Other

10 stars 4 forks source link

feat: extract dataset publish #159

Closed jjfrench closed 4 months ago

jjfrench commented 5 months ago

Summary: Summary of changes

Addresses VEDA-XX: Develop amazing new feature

Changes

DAG based off the /dataset/publish endpoint in the workflow api
- Checks that a collection doesn't already exists, generates a new template collection
- Override basic extents with spatial_extent and temporal_extent in dataset config
- Kicks off discovery DAG after ensuring that a collection exists

PR Checklist

[x] Ad-hoc testing - Deploy changes and test manually
[ ] Integration tests

smohiudd commented 4 months ago

Has this been deployed to SIT or another instance? Can you outline a testing plan to ensure that collections, items are being ingested into the catalog?