Please upload with the Codecov repository upload token to resolve issue

[amotl]

Hi Chris,

Problem

There are some problems with Codecov coverage uploads [1] we already experienced and resolved just recently at https://github.com/crate/crate-python/pull/451. I've also discovered it here, on behalf of the CI outcome of codecov/codecov-action#711.

[2022-10-24T20:43:15.807Z] ['error'] There was an error running the uploader:
Error uploading to [https://codecov.io:](https://codecov.io/) Error:
There was an error fetching the storage URL during POST: 404 - 
{'detail': ErrorDetail(string='Unable to locate build via Github Actions API.
Please upload with the Codecov repository upload token to resolve issue.', code='not_found')}

Solution

Can I ask you to add an item CODECOV_TOKEN to the secrets store at https://github.com/caronc/apprise/settings/secrets/actions, using the Codecov token which can be found at https://app.codecov.io/gh/caronc/apprise/settings?

With kind regards, Andreas.

[1] https://github.com/caronc/apprise/actions/runs/3316042793/jobs/5477373760#step:12:43

[caronc]

Sure thing, will address this in the morning

[caronc]

A bit late, but I think I added what you're asking in the right place. Let me know if you have any troubles.

[amotl]

Thank you Chris. I am not sure the whole thing works as intended, yet. At [1], it says

No token specified or token is empty

On the other project I switched to use the CODECOV_TOKEN environment variable, it says [2]:

Token found by environment variables

[1] https://github.com/caronc/apprise/actions/runs/3325043301/jobs/5497315337#step:12:37 [2] https://github.com/crate/crate-python/actions/runs/3316670557/jobs/5478729003#step:5:32

[amotl]

Hmm. Maybe this is the reason? https://github.com/caronc/apprise/blob/3078f35d9c4a3de1b04bec363f0b084e8a79970b/.github/workflows/tests.yml#L127

I will submit another patch to evaluate it.

[amotl]

codecov/codecov-action#719 apparently did not resolve it, there is still No token specified or token is empty at [1]. Can I humbly ask you to check the configuration at https://github.com/caronc/apprise/settings/secrets/actions again? Otherwise, we may have to merge codecov/codecov-action#719 first, and check again on a subsequent CI run.

[1] https://github.com/caronc/apprise/actions/runs/3325300235/jobs/5497841056#step:12:34

[caronc]

I tried adding the value elsewhere; I'm not sure if i got it right. I also tried to restart your build but it failed right away.

[amotl]

[1] still says:

No token specified or token is empty

Maybe we should merge codecov/codecov-action#719 first, and then check again on a subsequent PR. Thank you already!

[1] https://github.com/caronc/apprise/actions/runs/3353879851/jobs/5557026419#step:12:34

[caronc]

Done 👍

[amotl]

Yay! ^1 now says:

Token found by environment variables

[amotl]

Hmmm, again it says on recent CI runs at ^1:

No token specified or token is empty

Did you change anything?

[caronc]

Did you change anything?

No, I wonder if it depends on who owns the branch. Possibly this new way regular pull requests can't use the environment variable unless added as a maintainer? Or maybe there is an option where i can open this up for more? I wonder if there is a risk to this... (someone could do a PR with echo $CODECOV_TOKEN or something?

[amotl]

No, I wonder if it depends on who owns the branch. Possibly this new way regular pull requests can't use the environment variable unless added as a maintainer?

I think you may be right on this topic. For further details, see codecov/codecov-action#727.

[amotl]

On this matter, I would like to reference the resources [1] and [2-4]. [1] says:

Note: With the exception of GITHUB_TOKEN, secrets are not passed to the runner when a workflow is triggered from a forked repository.

[1] https://docs.github.com/en/actions/security-guides/encrypted-secrets#using-encrypted-secrets-in-a-workflow [2] https://securitylab.github.com/research/github-actions-preventing-pwn-requests/ [3] https://securitylab.github.com/research/github-actions-untrusted-input/ [4] https://securitylab.github.com/research/github-actions-building-blocks/

[amotl]

To make it properly work for PRs from forked repositories, I think we will indeed have to wait until https://github.com/codecov/feedback/issues/126 has been resolved. More news on this topic may also be shared by Tom at [1].

[1] https://community.codecov.com/t/unable-to-locate-build-via-github-actions-api-for-the-public-repository/3894

/cc @thomasrockhu, @thomasrockhu-codecov

[caronc]

Worst case, we can just roll back to Travis-CI. We just lose access to the windows testing. We could no branch the tests until the link you shared is resolved.

We can also just completely roll back several commits and just cherrypick the fixes outside this scope; there aren't that many. All of your work wouldn't be in vain, as we could reinstate it once your suggested approach has been made more stable.

It's my understanding that everything is working as intended so long as the owner or (assigned) collaborator creates the fork the runners access?

[amotl]

Dear Chris,

we will not have to roll back everything, I believe. Testing on CI works just fine, and also coverage tracking and reporting. There is only a slight issue when uploading the coverage reports to Codecov.

It is not that uploading never works, it actually works most of the time. There is only a slight flakyness when some rate-limiting hits the fan. I think it is acceptable for now, unless we observe any worsening on this very topic.

To reduce flakyness on the rare occasions where the rate-limiting prevents uploads, there is an advice to use the Codecov token for uploading, even on public repositories. The only drawback is that this does not work for PRs submitted from repository forks.

On this matter, when it really gets worse, I believe it would be sufficient to re-add Travis CI only for some slots, like Python 3.11 full + bare, only for the purpose of uploading coverage reports.

With kind regards, Andreas.