Airflow Development and Deployment

mike-gangl commented 5 months ago

Airflow Development and Deployment

Develop and deploy Airflow into the development, test, and SBG venues

Acceptance Criteria

deploy Airflow envrionments to venue-dev, venue-test, venue-ops and SBG-dev
Expose Airflow UI and rest calls to users through API gateway or HTTPD
create CWL operator task for airflow
WPS-T wrapper for airflow (deploy, execute)
demonstrate workflow executions (soundersips, SBG)

Stretch AC:

stretch: integrate airflow with deployment marketplace
stretch: enable node and pod scalability in airflow
stretch: EBS disk usage

Work Tickets

Link to work tickets required to implement the epic

Dependencies

Other epics or outside tickets required for this to work

[ ] TBC

Associated Risks

links to risk issues associated with this epic

[x] https://github.com/unity-sds/unity-project-management/issues/130

GodwinShen commented 5 months ago

@LucaCinquini please add and/or link your U-SPS work tickets to the acceptance criteria shown above. If you only have a handful to do in the current sprint, adding those for now is probably okay. It looks like you may have some tickets already in the queue (https://github.com/unity-sds/unity-sps/issues/28 and https://github.com/unity-sds/unity-sps/issues/32), so definitely add those ones in plus any additional ones that you identify as you go.

LucaCinquini commented 5 months ago

Here is the full SPS ticket board for this PI: https://github.com/orgs/unity-sds/projects/7/views/11

GodwinShen commented 5 months ago

Great, thanks!

unity-sds / unity-project-management