It takes time and effort to rerun everything locally. Like @SerVB proposed #82, we could move the workflow to GitHub Actions.
As the last step of building a given pull request, there should be an automated commit that updates all the reports (stuff inside python/box.tests/reports).
Be wary of an endless loop: the automation adds the commit, it triggers another build in the PR, which adds another commit, and so on. In theory this loop should be broken as soon as the N-th iteration regenerates the reports and the diff is empty, then I think committing will fail. This case has to be handled gracefully, best if just one build (per test task) is ran for each PR, it commits the reports and that's it.
It takes time and effort to rerun everything locally. Like @SerVB proposed #82, we could move the workflow to GitHub Actions.
As the last step of building a given pull request, there should be an automated commit that updates all the reports (stuff inside python/box.tests/reports).
Be wary of an endless loop: the automation adds the commit, it triggers another build in the PR, which adds another commit, and so on. In theory this loop should be broken as soon as the N-th iteration regenerates the reports and the diff is empty, then I think committing will fail. This case has to be handled gracefully, best if just one build (per test task) is ran for each PR, it commits the reports and that's it.