Factor out codecov upload

johannaengland commented 2 weeks ago

Closes #801.

Since upgrading the codecov action to version 4 uploading coverage reports without a token (so when creating a PR from a fork) is severely rate limited, which resulted often enough in our tests seemingly failing, even though it was only the coverage upload failing. (Reference: https://docs.codecov.com/docs/codecov-uploader#supporting-token-less-uploads-for-forks-of-open-source-repos-using-codecov)

Workflows that are triggered by other workflows are always run within the context of the main repository, instead of within the fork (except for workflows triggered by pushes to main in a fork, they are run within the context of the fork). This means that that way we can have access to the secret CODECOV_TOKEN and avoid the rate limitations.

Since we already upload our coverage results as artifacts in the Upload test reports step of the tests workflow, we decided to move the uploading coverage step to the same workflow where we publish the test results: publish-test-results. This workflow is triggered by the completion of the tests workflow, which means it is run within the context of the main repository.

Moving the uploading coverage step to the publish-test-results worked great in the way that we have access to the CODECOV_TOKEN, but due to the context switch codecov could not properly tell anymore which branch, commit and pull request that coverage information belongs to.

That is why we override the pull request, commit, base and branch information.

Getting the associated pull request number and base SHA is quite a bit more complicated and it involves saving them in a file, then uploading it as an artifact, then downloading that artifact and after validation using them in configuring codecov.

This also upgrades various actions used in these workflows. Reference for why we need to rename the artifacts after upgrading the upload artifacts action: https://github.blog/changelog/2023-12-14-github-actions-artifacts-v4-is-now-generally-available/