ReactionMechanismGenerator / RMG-tests

Continous Integration Testing Platform for RMG-Py
3 stars 14 forks source link

Complete Migration to GitHub Actions on RMG-Py #86

Closed JacksonBurns closed 1 year ago

JacksonBurns commented 1 year ago

TL;DR:

Migrate RMG-tests into RMG-Py and deprecate this repo. (1) if CI passes on RMG-Py, run the regression testing there by extending the GitHub action CI.yml (2) we can create our 'benchmark' model on a nightly basis and use GitHub artifacts to use it as comparison throughout the day without having to recreate it everytime we want to run regression (3) speed up the build by only building environment once (4) enable forked repos to work

The long version:

The unit testing has been moved into the GitHub actions in the RMG-Py repository, but the regression testing remains here. We should completely migrate this to also be part of the GitHub actions and deprecate RMG-tests.

Biggest limitation of the current workflow is that the regression testing will always fail on forked repos. This means that if want anyone outside of our groups to contribute, they needed to be added to the RMG user group and create a branch on RMG-Py itself. This basically means we cannot get other people to contribute.

Moving will also decrease the build time. Right now, we (1) build environment/install RMG to run unittests and (2) repeat this to run the regression tests (also using the slower conda instead of mamba).

We can achieve all the functionality of this repo in GitHub actions:

This would be a huge boost to maintainability -- all of the testing (unit tests + regression) would be consolidated in one actions file.

Request for Input

I am requesting some input on feasibility as well as my understanding of what this repo does. Please let me know if there is any part of the RMG-tests workflow I have missed or misunderstood. If there's something special about doing the regression testing here rather than in RMG-Py, I would like to discuss it. Tagging some of you who show up most often on the git blame for the relevant scripts: @sevyharris @KEHANG @Laxzal @rwest @nickvandewiele

JacksonBurns commented 1 year ago

The corresponding PR in RMG-Py has been merged (see this link) that implements all of the features in the TL;DR (except that MOPAC won't run on forked repos, which is a permanent limitation).

Regression testing now runs in ~2.5 hours, is automatically run nightly, and more rich comparisons can be made between versions because the built environment is distinct from the reference environment (RMG-tests would use the same dependencies regardless of the content of environment.yml).

I am going to transfer the relevant issues from RMG-tests to RMG-Py.