codecov pipeline not responding in PRs

[actionless]

i've noticed that lately in all of the currently opened and approved PRs (https://github.com/awesomeWM/awesome/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved), ex. https://github.com/awesomeWM/awesome/pull/3927 https://github.com/awesomeWM/awesome/pull/3930 https://github.com/awesomeWM/awesome/pull/3909

[Aire-One]

Some first investigations:

• Open some random PR from the list
• Go to the Checks tab
• In the Build & Test Block, click the codecov-lua5.3 menu item
• Unfold either Upload Lua code coverage report or Upload C code coverage report
• Find a rate limitation error message when Pinging Codecov

{'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 3122s.', code='throttled')}{'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 3119s.', code='throttled')}{'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 3117s.', code='throttled')}{'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 3115s.', code='throttled')}{'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 3113s.', code='throttled')}{'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 3111s.', code='throttled')}

[Aire-One]

Digging in our GitHub Workflow, we install some bash script from codecov (https://github.com/awesomeWM/awesome/blob/master/.github/workflows/main.yml#L243) to manage the coverage upload later (https://github.com/awesomeWM/awesome/blob/master/.github/workflows/main.yml#L324 and https://github.com/awesomeWM/awesome/blob/master/.github/workflows/main.yml#L347)

The bash script headers comment links to https://github.com/codecov/codecov-bash/blob/master/readme.md that is an archived repository with deprecation warnings.

I guess the next step for us is to move to the new uploader, https://about.codecov.io/blog/introducing-codecovs-new-uploader/.

Alternatively, there is now an Action by codecov https://docs.codecov.com/docs/github-2-getting-a-codecov-account-and-uploading-coverage. Maybe it would be easier to simply switch to using it 🤷

[sclu1034]

Changing deprecated tools is probably helpful in the long run, but I would suggest not to ignore error messages when they itself give you instructions on how to fix them:

Please upload with the Codecov repository upload token to resolve

A rate limit is server side, and unless they, for some weird reason, decided that the new tool gets to have a higher rate limit, it's going to run into the same issue.

Someone with access would have to log into codecov, get this repo's token, set that as a secret for actions, and configure the upload step to use it.

[actionless]

i never liked the dependency on those proprietary third-party services

i am thinking - could we just replace it to a bash one-liner which would compare coverage % in the report to nominal and return 0 (pipeline passing) or 1 (pipeline failing)?

[sclu1034]

At that point, you'd be better off removing the check altogether. codecov's main feature isn't the check mark. It's all the tools it gives you to analyze the data, both changes in coverage over time, and telling you the exact lines that are missing from coverage.

With your "bash one-liner", you'd know nothing about how the PR affects coverage. You wouldn't know if it changed coverage compared to master, and you wouldn't know which lines are or are not covered.

---

As I said above, the error message very clearly tells you what it wants you to do. Have either of you tried that?

[actionless]

i'd like to keep the check, as it helps to avoid pointless conversations on trying to convince people to cover their new features with some tests :) those tools are same available in locally generated html report, so i find the really valuable in codecov only the thing which compares PR coverage to master and check coverage of patch itself

ok, mb not a one-liner but a bit more of code, but generally it's not hard to compare the line numbers of the patch and in coverage report - but i was rather thinking if mb there are already existing cli tools for doing so

UPD: although comparing coverage to master branch would indeed require more effort, as it would mean to store coverage results from that branch somewhere, but i think it would be fine to check just coverage of the patch itself - would give more idea than no coverage check at all

---

i don't have the permissions to codecov

[Aire-One]

Have either of you tried that?

I don't have access to the repository secrets on GitHub, so I cannot add the API key for the Actions (even if I can read it on the codecov admin settings)

I think Actionless also don't have access to these GitHub settings. So we are doomed to wait for someone else...

[actionless]

So we are doomed to wait for someone else...

btw for some parts of the infra only @blueyed have the access, not even anyone else, so we have our website not being updated for more than a year 🤦

https://github.com/awesomeWM/awesome-www/issues/171

those problems getting to the point of being just ridiculous

[sclu1034]

we are doomed to wait for someone else... And this "someone else" shows up once a year, if ever. So I'd say you're doomed for a lot more than that. Not having access to codecov is probably among the least impactful things that could happen.

You might want to try to get a hold of the people that do still have access, especially for the repo. And I don't mean just one ping on GitHub every blue moon. blueyed has an email address in their profile, Elv seems still active on Reddit outside r/Awesome, etc...