[x] Create a test local build task, and a test GitHub Actions job, to run a test suite
[x] Trigger on push, on pull request, and on a cron schedule like rubberstamp
[x] Create a GitHub job that runs test suites, allowing the OS version to float. Here, build errors indicate when an upcoming OS release introduces breaking changes, in spite of any promises of backwards compatibility. As a bonus, continually testing against the very latest OS release, also helps to ensure that any OS package updates are included in testing.
[x] Detach futureproofing job statuses from blocking pull requests, as these are primarily informational. For emergency purposes, permit security ("audit") jobs to fail. But require all other GitHub Actions jobs to pass before pull requests are allowed to merge. https://stackoverflow.com/a/78072693/350106
As a consequence of triggering a test suite for continual OS and programming language updates, we're also granting a base layer of an automated test suite to more projects, that have previously relied on 100% manual tests.
Allowing security scans to fail helps in certain emergency scenarios. For example, an essential production hotfix should not be blocked by the presence of security bugs. A pull request that resolves some security bugs but not all security bugs, should not be blocked. A pull request that resolves some security bugs, but it hit with a surprise of yet more security bugs discovered, should not be blocked. Security scans should be clearly marked as pass vs. fail in pull request decorations, and it is up to repository owner(s) to examine any failures when determining whether to accept the pull request or ask contributors to resolve certain security bugs before merging.
Tasks:
https://github.com/mcandre/bao/settings/rules
As a consequence of triggering a test suite for continual OS and programming language updates, we're also granting a base layer of an automated test suite to more projects, that have previously relied on 100% manual tests.
Allowing security scans to fail helps in certain emergency scenarios. For example, an essential production hotfix should not be blocked by the presence of security bugs. A pull request that resolves some security bugs but not all security bugs, should not be blocked. A pull request that resolves some security bugs, but it hit with a surprise of yet more security bugs discovered, should not be blocked. Security scans should be clearly marked as pass vs. fail in pull request decorations, and it is up to repository owner(s) to examine any failures when determining whether to accept the pull request or ask contributors to resolve certain security bugs before merging.
Projects: