Open bhess opened 5 months ago
I think this is a good idea.
One thing we could do to pave the way is to smartly list all files that require tests to be run (basically *.c
, *.h
, and some generated .md
files?). If the changes don't involve at least one of these files, then nothing would be run.
I agree that this is a good idea. One "obvious" way to make CI more efficient would be to eliminate redundant runs (e.g., the same workflow being triggered by both "push" and "pull_request" on some PRs). I haven't yet found a clean way to do this, but perhaps somebody with more GitHub-specific expertise might.
IMO this is a great place to contribute for those who want to help out OQS / PQCA but don't feel comfortable working with cryptographic code.
I agree that this is a good idea. One "obvious" way to make CI more efficient would be to eliminate redundant runs (e.g., the same workflow being triggered by both "push" and "pull_request" on some PRs). I haven't yet found a clean way to do this, but perhaps somebody with more GitHub-specific expertise might.
@ryjones might have experience with this.
@SWilson4 do you mean on PR and merge, or the same job runs on both PR and the push of the PR to a branch?
@SWilson4 do you mean on PR and merge, or the same job runs on both PR and the push of the PR to a branch?
The latter. For example, see the "Linux and macOS tests" here: https://github.com/open-quantum-safe/liboqs/pull/1725/checks. These tests (and all others triggered to run on both push
and pull_request
events run twice on every push to an existing PR (at least for my PRs and presumably non-fork PRs).
@SWilson4 this stack overflow answer shows the way.
push:
branches:
- 'main'
- 'releases/**'
might be better
The outcome of today's OQS status call was that it would be good to have a document setting some guidelines / documenting best practices, saved as a Markdown file in the TSC repository. @bhess and @thb-sb to kick it off.
Thanks very much for documenting decisions such as this, @dstebila .
May I augment with the suggestion to link the file thus generated from a (new) "getting started contributing to OQS" write-up at www.openquantumsafe.org? Maybe off/in/at https://openquantumsafe.org/about/#getting-started (?)
In the absence of codified guidelines, here are some principles I tried to follow in https://github.com/open-quantum-safe/liboqs/pull/1909. I would suggest that we work toward following these in other projects as well.
I found reusable workflows to be very helpful for implementing all of these. These allow us to create "caller" workflows for each condition (push, pull request, release, schedule, etc) which trigger a subset of separate "callee" workflows (macOS tests, Linux tests, downstream tests, extended tests, etc). With this pattern we can get very fine-grained control over which tests run and when.
This issue is to suggest setting guidelines in OQS for resource-responsible use of CI.
Originally posted by @baentsch in https://github.com/open-quantum-safe/oqs-provider/issues/376#issuecomment-2006940977
I agree with the above and think some guidelines would help to avoid redundant CI runs, without hurting the overall test quality. This is not necessarily motivated by a lack of available resources (the LF indicated that more Github Actions resources can be made available to the projects, which I think is a good thing), but rather by the desire to make most efficient use of the available resources.
Any feedback welcome.