The complex package and data set dependencies of many of the apps make it difficult to write unit tests. This has started to cause some issues with making changes that end up breaking on very specific combinations of executors and apps.
Long term, we need integration tests. My vision for these, currently, is:
[ ] Conda environment build script for each app
[ ] Reference config file for each app that doesn't take too long to run
[ ] Reference config for each executor. Including ProcessPool, Dask, Ray, Parsl but not Globus Compute because it can't run locally or with all tasks.
[ ] Docker container with Conda environments built for each app and hosted on GitHub. This will also be helpful for local development
[ ] Integration jobs that run inside docker for each app with matrix for executor type
[ ] Run all integration tests weekly
[ ] Also run integration tests for specific app if that app is modified
The complex package and data set dependencies of many of the apps make it difficult to write unit tests. This has started to cause some issues with making changes that end up breaking on very specific combinations of executors and apps.
Long term, we need integration tests. My vision for these, currently, is: