search

unity-sds / unity-project-management

Container repo for project management (projects, epics, etc)
Apache License 2.0
0 stars 1 forks source link

Airflow Schedule #130

Closed mike-gangl closed 3 months ago

mike-gangl commented 5 months ago

Airflow Schedule

If the next generation SPS with Airflow falls severly behind schedule, we might need to switch back to the previous SPS implementation to fulfill the SBG use cases. Must demonstrate concrete progress (incuding first execution of SBG workflows) by end of February. Note that the SPS team has indicated the willingness to work 24/7 to NOT realize this risk.

Actions

Associated Objectives

links to risk issues associated with this epic

mike-gangl commented 5 months ago

We are going to move the 'checkin' of airflow progress up to mid-february. This gives us the best chance at understanding our odds to our scucess.

LucaCinquini commented 3 months ago

At this point we have demonstrated that Airflow is a very valid solution for being the SPS back-end technology. We have successfully demonstrated execution of the SBG pre-process workflow via CWL, and the SBG end-to-end is only crashing because of the lack of disk space (which will be remediated soon). Closing this ticket.

LucaCinquini commented 3 months ago

The e2e workflow has been successfully demonstrated on the SPS/Airflow system using a single CWL workflow. Closing this ticket as the risk did not materialize.