Configure Airflow for use

Azure / osdu-infrastructure

OSDU Azure Infrastructure as Code

Apache License 2.0

42 stars 39 forks source link

Configure Airflow for use #138

Closed danielscholl closed 4 years ago

danielscholl commented 4 years ago

Airflow is required in order to support the Ingest Workstream. An initial PR #130 was submitted which will need to be broken up into smaller tasks and PR's and tracked here in this Issue.

[x] Add Support for Storage File Shares and Storage Queues in the Storage Module.
[x] Add (DAG) Storage into the osdu-mvp architecture.
[x] Add (DAG) Storage into the osdu-r3-resources architecture.
[x] Update the osdu-common helm chart kv-secrets.yaml to add in secrets for Reds, Postgres, and DAG Storage.

danielscholl commented 4 years ago

@catweisun Storage for Airflow is added to OSDU-mvp architecture.

imran-siddique commented 4 years ago

@NickKarsky - Where are we tracking the file service work?

danielscholl commented 4 years ago

This issue has been completed as part of osdu-mvp architecture.