flatironinstitute / disBatch

Tool to distribute a list of computational tasks over a pool of compute resources. The pool can grow or shrink.
Apache License 2.0
39 stars 8 forks source link

Add basic tests. Run the ssh-mode test on GitHub Actions. #27

Closed lgarrison closed 1 year ago

lgarrison commented 1 year ago

This PR adds a basic test, which touches some files then checks that those files exist. There are two copies of this test: one for ssh mode, and one for Slurm. While for now the only thing different about the tests is the disBatch invocation, I've kept them separate because we might want to add more functionality to the Slurm test in the future. The ssh-mode is tested on GitHub Actions for Python versions 3.6 to 3.10.

The Slurm test can only be run locally for now, but in theory it would be possible to set up a GitHub Action that installs Slurm and runs the test.

njcarriero commented 1 year ago

Looks good. Two quick questions:

lgarrison commented 1 year ago

Sure, I'll add Python 3.11. I haven't been including it in my other projects because many upstream dependencies don't support it yet, but disBatch doesn't have any upstream dependencies!

There's no need to clean anything up on the GitHub runners, although it currently doesn't clean up local temp dirs to aid debugging/manual inspection. Maybe I'll have it clean up after itself on a successful run though...