olcf / olcf-test-harness

OLCF Test Harness
https://olcf.github.io/olcf-test-harness/
Other
12 stars 4 forks source link

Feature Request: Create a Python API for launching the OLCF Test Harness #154

Open hagertnl opened 11 months ago

hagertnl commented 11 months ago

Instead of passing around a bunch of runtests.py commands and rgt.inp files, it may be easier to create a Python API that allows users to import an API module and run, similar to the following:

from olcf_test_harness_pyapi import oth_launch

# This call blocks until an apptest has been submitted to the queue
oth_launch.launch_apptest(app="osu-microbenchmarks", test="test_startup", path="/path/to/tests/applications", mode=['start', 'stop'])

We can grow the API as needed from there, but I think this would be very helpful. Then we could possibly have Python scripts that we can create and use instead of many different rgt.inp files.

hagertnl commented 11 months ago

Commenting to keep track of some more API functions that would be nice:

hagertnl commented 11 months ago

Another use case: a utility named check_available_apps.py, which would act like this:

$ check_available_apps.py --path /path/to/test/applications
App: LAMMPS
Git Branch/Commit: main/a0b1c2
Tests:
- test1
- test2
- test3
App: GROMACS
...

This would be helpful to verify --mode checkout results.