When a new pull request is submitted or is updated by a new commit, automatically trigger custom Github Actions workflow to test the build of specified models before allowing a merge.
Is your feature request related to a problem?
There is no capacity for automated tests to be ran in DART. A lot of the test cases implemented must be run and checked manually by a developer, taking away their time (especially if trying to get enough code coverage on tests).
Describe your preferred solution
Use and implement Github Actions workflows that get triggered upon new pull-requests and subsequent commits to build, run, or test specific models or programs in DART.
Ideally, the Github Actions workflows are able to be easily expanded upon by other developers that want to implement their own test methods for the features that they are working on.
Workflows triggered will run on Github's 'runner' compute instances that are available for use. The workflow will then report back its results in the related pull request and in the Actions tab of the DART repo.
Examples of test workflows that can be specified via Github Actions:
automatically build and run lorenz_96 with MPI
automatically build and run an obs_converter model
automatically run specified test cases inside of developer_tests
automatically build and run everything
In the long run, implementing automated tests and continuous integration infrastructure will help save developer time by catching bugs earlier than when caught in the future when its already in production code.
Describe any alternatives you have considered
An alternative solution to using Github Actions is to have scheduled cronjobs on Cheyenne listen to the DART repo via the Github API and automatically run tests as a job submitted to Cheyenne. It may be best to have both Github Actions and cronjobs on Cheyenne to support larger test jobs (such as build everything). Github Actions' runner compute instances are limited to 2 cores (unless NCAR github org is of a higher tier, which will then have access to larger runners).
Use case
When a new pull request is submitted or is updated by a new commit, automatically trigger custom Github Actions workflow to test the build of specified models before allowing a merge.
Is your feature request related to a problem?
There is no capacity for automated tests to be ran in DART. A lot of the test cases implemented must be run and checked manually by a developer, taking away their time (especially if trying to get enough code coverage on tests).
Describe your preferred solution
Use and implement Github Actions workflows that get triggered upon new pull-requests and subsequent commits to build, run, or test specific models or programs in DART. Ideally, the Github Actions workflows are able to be easily expanded upon by other developers that want to implement their own test methods for the features that they are working on.
Workflows triggered will run on Github's 'runner' compute instances that are available for use. The workflow will then report back its results in the related pull request and in the Actions tab of the DART repo.
Examples of test workflows that can be specified via Github Actions:
developer_testsIn the long run, implementing automated tests and continuous integration infrastructure will help save developer time by catching bugs earlier than when caught in the future when its already in production code.
Describe any alternatives you have considered
An alternative solution to using Github Actions is to have scheduled cronjobs on Cheyenne listen to the DART repo via the Github API and automatically run tests as a job submitted to Cheyenne. It may be best to have both Github Actions and cronjobs on Cheyenne to support larger test jobs (such as build everything). Github Actions' runner compute instances are limited to 2 cores (unless NCAR github org is of a higher tier, which will then have access to larger runners).