Uninett / zino

Zino 2.0 - Network state monitor for research networks
Apache License 2.0
3 stars 4 forks source link

Factor out codecov upload #230

Closed johannaengland closed 1 month ago

johannaengland commented 1 month ago

Closes #216.

Since upgrading the codecov action to version 4 uploading coverage reports without a token (so when creating a PR from a fork) is severely rate limited, which resulted often enough in our tests seemingly failing, even though it was only the coverage upload failing. (Reference: https://docs.codecov.com/docs/codecov-uploader#supporting-token-less-uploads-for-forks-of-open-source-repos-using-codecov)

Workflows that are triggered by other workflows are always run within the context of the main repository, instead of within the fork (which workflows triggered by pull requests for example can be). This means that that way we can have access to the secret CODECOV_TOKEN and avoid the rate limitations.

Since we already upload our coverage results as artifacts in the Upload test reports step of the tests workflow, we decided to move the uploading coverage step to the same workflow where we publish the test results: publish-test-results. This workflow is triggered by the completion of the tests workflow, which means it is run within the context of the main repository.

Moving the uploading coverage step to the publish-test-results worked great in the way that we have access to the CODECOV_TOKEN, but due to the context switch codecov could not properly tell anymore which branch, commit and pull request that coverage information belongs to.

That is why we override the pull request, commit and branch information.

Getting the associated pull request number is quite a bit more complicated and it involves saving that number in a file, then uploading it as an artifact, then downloading that artifact and after validation setting that as the pull request number for codecov.

I have tested this with four different scenarios:

  1. Push against main, no associated pull request number (tests, upload)
  2. Pull request with passing tests (tests, upload)
  3. Pull request with failing tests (tests, upload)
  4. Pull request that overrides the variable written to the file that will be uploaded (tests, upload)

This also upgrades various actions used in these workflows. Reference for why we need to rename the artifacts after upgrading the upload artifacts action: https://github.blog/changelog/2023-12-14-github-actions-artifacts-v4-is-now-generally-available/

github-actions[bot] commented 1 month ago

Test results

    3 files      3 suites   51s :stopwatch: 318 tests 318 :heavy_check_mark: 0 :zzz: 0 :x: 954 runs  952 :heavy_check_mark: 2 :zzz: 0 :x:

Results for commit 0ebbddb1.

:recycle: This comment has been updated with latest results.

johannaengland commented 1 month ago

Also just to have it documented somewhere: In a discussion on how to get the pull request number since it is not easily accessible when running in a workflow triggered by workflow_run it was mentioned to check how the action publish-test-results handles it. I figured out it is how we ended up doing it: uploading an artifact that has all information about the event that triggered the workflow that will trigger the workflow that runs the publish-test-results action. That artifact is then used later to determine all the details like pull request number, etc. The upload can be seen here: https://github.com/EnricoMi/publish-unit-test-result-action/blob/master/.github/workflows/ci-cd.yml#L175

sonarcloud[bot] commented 1 month ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud