Open thomasrockhu-codecov opened 1 week ago
In the repository settings page in codecov.io, you can set the ability for Codecov to receive a coverage report from ANY souce.
Where can I find it for a "single repository"? I can find only a global "Token authentication" option for the whole organization under /org-upload-token
. What about organizations where some repositories are public and other are internal/private?
Related PR: https://github.com/signalfx/splunk-otel-go/pull/3496
I also noticed that the v4
action seems still able to work tokenless on forks without enabling "Not required" token authentication. This does not apply to v5
.
From https://docs.codecov.com/docs/codecov-tokens#uploading-without-a-token:
the upload is for a commit that is on an unprotected branch (like forkname:main)
I find this description not clear.
main
branch will not work if the branch is protected?main
) in upstream?Besides, both of these scenarios are not well suited for public open-source repositories.
(2) does not seem to be true.
Reference PR: https://github.com/open-telemetry/opentelemetry-go/pull/5979
I noticed that for forks it does not seem to be a problem for v4
.
@pellared
Where can I find it for a "single repository"? I can find only a global "Token authentication" option for the whole organization under /org-upload-token. What about organizations where some repositories are public and other are internal/private?
This was my mistake. It is in fact for the whole organization. However, private repositories will always need a token to authenticate. I have updated this issue and the README. I apologize for the misinformation.
I also noticed that the v4 action seems still able to work tokenless on forks without enabling "Not required" token authentication. This does not apply to v5.
That is strange, can you link me to corresponding CI runs here? It shouldn't be different.
the upload is for a commit that is on an unprotected branch (like forkname:main)
To clear things up, we do not mean protected
from a GitHub perspective.
For public repositories, a token is required to send uploads to commits on protected branches. A protected branch corresponds to an actual branch in your repository (like main or master). An unprotected branch is any branch with a colon-separated prefix on it (like forkname:main or pr300:master).
I also noticed that the v4 action seems still able to work tokenless on forks without enabling "Not required" token authentication. This does not apply to v5.
That is strange, can you link me to corresponding CI runs here? It shouldn't be different.
Putting GH run logs from https://github.com/open-telemetry/opentelemetry-go/pull/5964 PR (https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858) at the end of this comment. At that time the org had token authentication option set to "Required" .
Possibly fixed by https://github.com/codecov/codecov-action/pull/1650?
Run codecov/codecov-action@v4.6.0
with:
file: ./coverage.txt
verbose: true
env:
DEFAULT_GO_VERSION: ~1.[2](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:2)3.0
CODECOV_TOKEN:
eventName: pull_request
baseRef: open-telemetry:main | headRef: mmorel-[3](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:3)5:golangci-lint/perfsprint
==> Fork detected, tokenless uploading used
==> linux OS detected
https://cli.codecov.io/latest/linux/codecov.SHA256SUM
Received SHA256SUM [4](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:4)2803b7cc22e28e12a02c8443[5](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:5)cd6f6fbde4194[6](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:6)fb15038f0791669906508a03 codecov
Received SHA256SUM signature -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEJwNOf9uFDgu8LGL/gGuyiu13mGkFAmc2O0wACgkQgGuyiu13
mGnELw/6A6HAqhc/Jcyg49/Vgd5adMkhJnykeMIrpm/jnXMV2zTpPqCHnd9iz[7](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:7)zo
NtaDsIKxfwkS2DZ5Sm3LH44/4VFSegjffzC5FjueMoo[8](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:9)eiWtPVDymoRK2YsMjn6i
J/B+MC+ld+U6Sn2nD2HdNGQHjYtHYrpDshzGKEcEnpj/W[9](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:10)2+ZN/OK6N5p55nkjbV
Ji7KY2jeySTN49TtYbdgRTnWOfz6QA7a7iHKHAPeUavEveVXYT/kuD88w89rTs7/
+c+wa8mxXyGdMJ+ytff9KV1hKPy1ksyQDUB/hHjHQWbqCeH2d8BDs8S1g/VcRW2k
S2VeE2JCOz05EBaerPLIKKTvetisI1hoinHEa2+k0OtkVdLfy2hmqlzc4EZkxIFk
7uMOEfZp8t0ZGczzteON7Omd/VwOFRPob85kkAJXxbIHyiVuD7XtUl5eVO3fQQsV
vbrMJK/io9vDaRhiYLiHBHO2ZhWEi22HpnYS30U2icV0dPTC887CdIScUJmrQA24
nRSD/QWvIFuQwrnRpebQI3nDroZF7cwpCcppDAZ8CNvT7+EWptYq9PXT4vKXI3R8
X7yRrgF20ifdmBV1yDnWlDJ9KD6c5PeZj1+xloNTmeg81JHlyWdwJRlCYYVxAuOV
fGf+gGq1/x0OXAaSvb3iriYNBEpL3psfjcrcKr9HgCa+vnh7lEE=
=MFsR
-----END PGP SIGNATURE-----
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: /home/runner/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" imported
gpg: Total number processed: 1
gpg: imported: 1
gpg: Signature made Thu Nov 14 18:02:52 2024 UTC
gpg: using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
gpg: Good signature from "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 2703 4E7F DB85 0E0B BC2C 62FF 806B B28A ED77 9869
==> Uploader SHASUM verified (42803b7cc22e28e12a02c84435cd6f6fbde41946fb15038f0791669906508a03 codecov)
==> Running version latest
==> Running version v9.0.4
==> Running git config --global --add safe.directory /home/runner/work/opentelemetry-go/opentelemetry-go
/usr/bin/git config --global --add safe.directory /home/runner/work/opentelemetry-go/opentelemetry-go
==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4.6.0/dist/codecov -v create-commit'
/home/runner/work/_actions/codecov/codecov-action/v4.6.0/dist/codecov -v create-commit --git-service github -B mmorel-35:golangci-lint/perfsprint -C 6f2881a022d1d4a8b2f74dbafbd34b994e17983e
info - 2024-11-14 18:02:55,352 -- ci service found: github-actions
debug - 2024-11-14 18:02:55,355 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
debug - 2024-11-14 18:02:55,357 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
warning - 2024-11-14 18:02:55,357 -- No config file could be found. Ignoring config.
debug - 2024-11-14 18:02:55,358 -- No codecov_yaml found
debug - 2024-11-14 18:02:55,358 -- Starting create commit process --- {"verbose": true, "auto_load_params_from": null, "codecov_yml_path": null, "enterprise_url": null, "version": "cli-9.0.4", "command": "create-commit", "git_service": "github", "branch": "mmorel-35:golangci-lint/perfsprint", "commit_sha": "6f2881a022d1d4a8b2f74dbafbd34b994e17983e", "parent_sha": null, "pull_request_number": "5964", "fail_on_error": false, "slug": "open-telemetry/opentelemetry-go"}
info - 2024-11-14 18:02:55,358 -- The PR is happening in a forked repo. Using tokenless upload.
info - 2024-11-14 18:02:55,949 -- Process Commit creating complete
debug - 2024-11-14 18:02:55,949 -- Commit creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=202, text='{\"status\":\"queued\"}\\n')"}
==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4.6.0/dist/codecov -v create-report'
/home/runner/work/_actions/codecov/codecov-action/v4.6.0/dist/codecov -v create-report --git-service github -C 6f2881a022d1d4a8b2f74dbafbd34b994e17983e
info - 2024-11-14 18:02:56,663 -- ci service found: github-actions
debug - 2024-11-14 18:02:56,666 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
debug - 2024-11-14 18:02:56,668 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
warning - 2024-11-14 18:02:56,668 -- No config file could be found. Ignoring config.
debug - 2024-11-14 18:02:56,669 -- No codecov_yaml found
debug - 2024-11-14 18:02:56,669 -- Starting create report process --- {"verbose": true, "auto_load_params_from": null, "codecov_yml_path": null, "enterprise_url": null, "version": "cli-9.0.4", "command": "create-report", "git_service": "github", "commit_sha": "6f2881a022d1d4a8b2f74dbafbd34b994e17983e", "code": "default", "pull_request_number": "5964", "fail_on_error": false, "slug": "open-telemetry/opentelemetry-go"}
info - 2024-11-14 18:02:57,120 -- Process Report creating complete
debug - 2024-11-14 18:02:57,120 -- Report creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=202, text='{\"status\":\"queued\"}\\n')"}
info - 2024-11-14 18:02:57,120 -- Finished creating report successfully --- {"response": "{\"status\":\"queued\"}\n"}
==> Running command '/home/runner/work/_actions/codecov/codecov-action/v4.6.0/dist/codecov -v do-upload'
/home/runner/work/_actions/codecov/codecov-action/v4.6.0/dist/codecov -v do-upload -f ./coverage.txt --git-service github -C 6f2881a022d1d4a8b2f74dbafbd34b994e17983e
info - 2024-11-14 18:02:57,835 -- ci service found: github-actions
debug - 2024-11-14 18:02:57,838 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
debug - 2024-11-14 18:02:57,840 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.NoVersioningSystem'>
warning - 2024-11-14 18:02:57,840 -- No config file could be found. Ignoring config.
debug - 2024-11-14 18:02:57,841 -- No codecov_yaml found
debug - 2024-11-14 18:02:57,842 -- Starting upload processing --- {"verbose": true, "auto_load_params_from": null, "codecov_yml_path": null, "enterprise_url": null, "version": "cli-9.0.4", "command": "do-upload", "git_service": "github", "commit_sha": "6f2881a022d1d4a8b2f74dbafbd34b994e17983e", "report_code": "default", "network_root_folder": "/home/runner/work/opentelemetry-go/opentelemetry-go", "files_search_root_folder": "/home/runner/work/opentelemetry-go/opentelemetry-go", "files_search_exclude_folders": [], "disable_search": false, "disable_file_fixes": false, "build_code": "11842872629", "build_url": "https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629", "job_code": "ci", "name": "11842872629", "branch": "golangci-lint/perfsprint", "pull_request_number": "5964", "env_vars": {}, "flags": [], "plugin_names": ["xcode", "gcov", "pycoverage"], "dry_run": false, "use_legacy_uploader": false, "handle_no_reports_found": false, "report_type": "coverage", "network_filter": null, "network_prefix": null, "gcov_args": null, "gcov_ignore": null, "gcov_include": null, "gcov_executable": null, "swift_project": null, "fail_on_error": false, "slug": "open-telemetry/opentelemetry-go"}
debug - 2024-11-14 18:02:57,842 -- Selected preparation plugins --- {"selected_plugins": ["<class 'codecov_cli.plugins.xcode.XcodePlugin'>", "<class 'codecov_cli.plugins.gcov.GcovPlugin'>", "<class 'codecov_cli.plugins.pycoverage.Pycoverage'>"], "cli_config": {}}
debug - 2024-11-14 18:02:57,842 -- Running preparation plugin: <class 'codecov_cli.plugins.xcode.XcodePlugin'>
debug - 2024-11-14 18:02:57,842 -- Running xcode plugin...
warning - 2024-11-14 18:02:57,842 -- xcrun is not installed or can't be found.
debug - 2024-11-14 18:02:57,842 -- Running preparation plugin: <class 'codecov_cli.plugins.gcov.GcovPlugin'>
debug - 2024-11-14 18:02:57,842 -- Running gcov plugin...
warning - 2024-11-14 18:02:57,843 -- No gcov data found.
debug - 2024-11-14 18:02:57,843 -- Running preparation plugin: <class 'codecov_cli.plugins.pycoverage.Pycoverage'>
warning - 2024-11-14 18:02:57,843 -- coverage.py is not installed or can't be found.
debug - 2024-11-14 18:02:57,843 -- Collecting relevant files
warning - 2024-11-14 18:02:57,848 -- Some files were not found --- {"not_found_files": ["coverage.txt"]}
info - 2024-11-14 18:02:57,850 -- Found 1 coverage files to report
info - 2024-11-14 18:02:57,850 -- > /home/runner/work/opentelemetry-go/opentelemetry-go/coverage-artifacts-~1.23.0/coverage.txt
debug - 2024-11-14 18:02:57,850 -- Selected uploader to use: <class 'codecov_cli.services.upload.upload_sender.UploadSender'>
debug - 2024-11-14 18:02:57,862 -- Sending upload request to Codecov
info - 2024-11-14 18:02:58,251 -- Your upload is now processing. When finished, results will be available at: https://app.codecov.io/github/open-telemetry/opentelemetry-go/commit/6f2881a022d1d4a8b2f74dbafbd34b994e17983e
debug - 2024-11-14 18:02:58,251 -- Upload request to Codecov complete. --- {"response": {"raw_upload_location": "https://storage.googleapis.com/codecov/shelter/github/open-telemetry%3A%3A%3A%3Aopentelemetry-go/6f2881a022d1d4a8b2f74dbafbd34b994e17983e/a90b1c64-[10](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:11)2f-4a61-b194-f9a7c8e7ab97.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJWQHUGIBILH4J7Q6ZUSCIFNEOLYSNDS7L3B4N5SIBQ2J4YLYE5CRFCD%2F2024[11](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:12)14%2Fus%2Fs3%2Faws4_request&X-Amz-Date=20241114T180258Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=357d8a7c032be34cfa5cdb5428f94652af358d709311791b478ff22fe4c2daf2", "url": "https://app.codecov.io/github/open-telemetry/opentelemetry-go/commit/6f2881a022d1d4a8b2f74dbafbd34b994e17983e"}}
debug - 2024-11-[14](https://github.com/open-telemetry/opentelemetry-go/actions/runs/11842872629/job/33002565858#step:3:15) 18:02:58,251 -- Sending upload to storage
info - 2024-11-14 18:02:58,405 -- Process Upload complete
debug - 2024-11-14 18:02:58,405 -- Upload result --- {"result": "RequestResult(error=None, warnings=[], status_code=200, text='')"}
To clear things up, we do not mean protected from a GitHub perspective.
For public repositories, a token is required to send uploads to commits on protected branches. A protected branch corresponds to an actual branch in your repository (like main or master). An unprotected branch is any branch with a colon-separated prefix on it (like forkname:main or pr300:master).
Does it mean that even with authentication option set to "Not required" we still need a token here: https://github.com/open-telemetry/opentelemetry-go/pull/5979? If so then, I am not following what has changed regarding token authentication 🤷
My dependabot just updated codecov from 4 to 5: https://github.com/maplibre/maplibre-gl-js/pull/5050 I no longer see the codecov message in the PR thread. Is this expected? Is this a bug? The following is the relevant PR: https://github.com/maplibre/maplibre-gl-js/pull/5051
In both the dependabot PR and the newly created one I don't see codecov coverage message. Please advise or update the migration instructions.
@HarelM, see https://github.com/maplibre/maplibre-gl-js/actions/runs/11853207459/job/33032875399#step:8:168
error - 2024-11-15 08:58:02,906 -- Upload failed: {"message":"Token required because branch is protected"}
You are not using the fail_ci_if_error: true
option so the action passed with an error.
However, private repositories will always need a token to authenticate
@thomasrockhu-codecov, if so then the documentation should improved as currently it does not have any sense
If I guess correctly this option only applies for public repositories to set whether codecov accepts tokenless reports or not (when the report is send from the actual repository and not from a fork). But maybe my guess is bad.
Is the previous token configuration not valid anymore? There is a codecov token configured for this repo and PR...
Bottom line, I think the migration/upgrade instructions can be improved given the issues that were recently opened and the discussion here. I would also argue that a CI that can fail should not have a default of continue with any notification and that it should fail the CI pipeline unless told otherwise, but that might just be me...
I have another general question about the v5. I've read the diff, and I'm concerned about the change from a node-based action, that is supposed to be cross-platform (until it doesn't), whilst the new action is composite-based, and relies heavily on bash, and other bash features. How well does it work on other platforms? I still expect some success on Windows GitHub-hosted runners, as they include a git bash, but what about self-hosted runners?
For macOS, on a project I'm working on we couldn't use code coverage yet, as the uploading action relied on some gpg binaries being available with homebrew, but for our software to build, we need to remove some installed software to use another toolchain, and even though the same software is installed, the path was hardcoded to the original path. Will v5 behave differently?
@pellared yes, I'm going to update the documentation today as it seems that it's quite unclear. Would you be able to see if 5.0.1
fixes the issues you were having before?
If I guess correctly this option only applies for public repositories to set whether codecov accepts tokenless reports or not (when the report is send from the actual repository and not from a fork). But maybe my guess is bad.
That is correct, I need to make an update to the product team for this change, but it is only relevant to public repositories.
@HarelM I saw this PR you made. I just pushed 5.0.1
, but not sure if that will change your issue. Just so I'm clear, the issue is that you're not getting a PR comment from Codecov anymore? Or is it also that you're not getting status checks too?
Bottom line, I think the migration/upgrade instructions can be improved given the issues that were recently opened and the discussion here.
💯, although it should have been more clear before we launched, I will spend my day improving the pieces that I can.
I would also argue that a CI that can fail should not have a default of continue with any notification and that it should fail the CI pipeline unless told otherwise, but that might just be me...
This is definitely something we have discussed a lot within the team. The gist of it is that we have users that have extremely long-running CI builds that don't necessarily rely on Codecov to merge. If there is a failure at that stage, it can be extremely painful for users. As a result, Codecov has always made this a non-blocking step by default in case of our own downtime or irregularities.
We have been building and have built a lot of safeguards to prevent such issues and are leaning towards flipping it in a future major release of this Action
@echoix, our expectation is that the v5
release will work on as many runners as possible. There are definitely some OSes (e.g. FreeBSD) that we are currently working on supporting, but it is possible that we made some mistakes for existing runners in transferring over to a composite action. It is my highest priority currently to ensure v5
has feature parity+ to v4
As for gpg
, we are working on moving to ssh
validation as it seems to be more accessible to all developers. It will likely be released as a minor version.
@pellared yes, I'm going to update the documentation today as it seems that it's quite unclear. Would you be able to see if
5.0.1
fixes the issues you were having before?
Looks to be fixed: https://github.com/signalfx/splunk-otel-go/actions/runs/11857558947/job/33046215151#step:5:159. Thanks.
@HarelM I saw this PR you made. I just pushed 5.0.1, but not sure if that will change your issue. Just so I'm clear, the issue is that you're not getting a PR comment from Codecov anymore? Or is it also that you're not getting status checks too?
@thomasrockhu-codecov I didn't see the status checks in the renovate PR to update this, as we don't enable PR comments, and only look at the status checks there. That made me uncertain if the upload and tokens worked, I was suspicious of the action update. I have my fork that also has renovate enabled, and also codecov configured with a separate token, and was able to merge it to my main, and in there, once merged, the status checked were there.
@echoix got it, I think I found the root cause, working on a fix now
@echoix ok, I pushed 5.0.2
which should hopefully fix this issue
@echoix ok, I pushed
5.0.2
which should hopefully fix this issue
Let's see that in action once https://github.com/OSGeo/grass/pull/4704 updates itself
Seems like it returned to work, thanks! https://github.com/maplibre/maplibre-gl-js/pull/5054#issuecomment-2480757567
It seems that a private repo workflow using another private repo workflow within the same organization is not supported. (with 5.x)
Our actions return with ==> Fork detected, tokenless uploading used
but it's not a fork and tokens are provided...
==> Running create-commit
./codecov create-commit --fail-on-error --git-service github --sha SHASHASHASHASHASHASHASHA
info - 2024-11-18 13:02:58,465 -- ci service found: github-actions
info - 2024-11-18 13:02:58,488 -- The PR is happening in a forked repo. Using tokenless upload.
info - 2024-11-18 13:02:58,783 -- Process Commit creating complete
error - 2024-11-18 13:02:58,784 -- Commit creating failed: {"message":"Token required - not valid tokenless upload"}
==> Failed to create-commit
Exiting...
Error: Process completed with exit code 1.
Code repo workflow:
<snip>
jobs:
test-php:
name: PHPUnit tests
uses: <org>/github-workflows/.github/workflows/unittest-php.yml@main
with:
<snip>
secrets:
<snip>
codecov-token: ${{ secrets.CODECOV_TOKEN }}
<snip>
<org>/github-workflows/.github/workflows/unittest-php.yml
contains some jobs
steps and local actions
composite steps.
<snip>
jobs:
phpunit:
-
uses: <org>/github-workflows/.github/actions/action-unittest-php@main
with:
<snip>
codecov-token: ${{ secrets.codecov-token }}
<snip>
<org>/github-workflows/.github/actions/action-unittest-php
name: 📤 Upload coverage reports to Codecov
uses: codecov/codecov-action@5c47607acb93fed5485fdbf7232e8a31425f672a # 5.0.2
with:
files: clover.xml
fail_ci_if_error: true
name: codecov-${{ inputs.php-version }}
env:
CODECOV_TOKEN: ${{ inputs.codecov-token }}
why this way? one to have centralized control over the workflows, second to have less duplicate workflow steps merged in composite actions.
@MarcHagen I'm also seeing similar behavior upon upgrading to codecov-action@v5 from v4. I have opened https://github.com/codecov/codecov-action/issues/1671 to track the issue.
@MarcHagen @Cardds I pushed 5.0.3
which I think will fix the issue. Would you try and see if that helps?
Note to self that #1671 is tracking the issue for @Cardds and @MarcHagen
v5 Release
v5
of the Codecov GitHub Action will use the Codecov Wrapper to encapsulate the CLI. This will help ensure that the Action gets updates quicker.Migration Guide
The
v5
release also coincides with the opt-out feature for tokens for public repositories. In theGlobal Upload Token
section of the settings page of an organization in codecov.io, you can set the ability for Codecov to receive a coverage reports from any source. This will allow contributors or other members of a repository to upload without needing access to the Codecov token. For more details see how to upload without a token.The following arguments have been added:
binary
gcov_args
gcov_executable
gcov_ignore
gcov_include
report_type
skip_validation
swift_project
You can see their usage in the
action.yml
file.If you are having troubles with the migration, please open a new issue and tag @codecov/report-upload