search

ejjordan / airflowHPC

2 stars 1 forks source link

Use case examples to test in Airflow #33

Open andre-merzky opened 1 month ago

andre-merzky commented 1 month ago

I would like to make sure that the ScaleMS use cases are covered by the current Airflow executor and operator semantics. Beyond the REXEE_* dags, what other dags should be used to confirm viability of the stack?

ejjordan commented 1 month ago

I have created a number of DAGs for this purpose. In particular, the test_* DAGs should be helpful to this end, with tests for resources, MPI/OpenMP, and PyTorch. Additionally the gmx_multi DAG is very useful for testing purposes.