jonescompneurolab / hnn-core

Simulation and optimization of neural circuits for MEG/EEG source estimates
https://jonescompneurolab.github.io/hnn-core/
BSD 3-Clause "New" or "Revised" License
50 stars 50 forks source link

Debug codecov issues #759

Closed gtdang closed 2 months ago

gtdang commented 2 months ago

Debugging GitHub actions for uploading codecov report. Unittest actions have been failing since switching to using the codecov-action v4 in #758 .

Codecov uploads have sporadically been failing due to rate-limiting of 'tokenless' uploads. It is not actually tokenless, see explanation below.

If Codecov detects that reports are being uploaded from a fork of an open source repository, Codecov will automatically switch to tokenless uploading...Token-less uploads still use a token to upload to Codecov - just that it's Codecov's Github token instead of the project's own. As such, uploading to Codecov via tokenless (whether it's via forks, or by an older version of the Codecov action) ends up consuming from the shared request pool of a lot of Codecov users, instead of your own. Therefore, there may be cases when uploading to Codecov via tokenless fails because Codecov is being rate limited. Source

This issue has been documented by others: https://github.com/codecov/feedback/issues/358 https://github.com/codecov/feedback/issues/301 https://github.com/codecov/feedback/issues/112

codecov-commenter commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 60.29%. Comparing base (d28ddb7) to head (c084c47). Report is 36 commits behind head on master.

:exclamation: Current head c084c47 differs from pull request most recent head dec481a. Consider uploading reports for the commit dec481a to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #759 +/- ## =========================================== - Coverage 92.67% 60.29% -32.39% =========================================== Files 27 27 Lines 4928 4969 +41 =========================================== - Hits 4567 2996 -1571 - Misses 361 1973 +1612 ``` | [Flag](https://app.codecov.io/gh/jonescompneurolab/hnn-core/pull/759/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jonescompneurolab) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/jonescompneurolab/hnn-core/pull/759/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jonescompneurolab) | `60.29% <ø> (?)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jonescompneurolab#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

gtdang commented 2 months ago

One option is changing the setting fail_ci_if_error = false. This will not fail the test if the coverage isn't uploaded. There just won't be a coverage report.

gtdang commented 2 months ago

Bash implementation doesn't fail the workflow. Seems to be an error during upload and it's requesting a token. I'm not sure if a new report was actually generated...

Screenshot 2024-05-07 at 10 07 13 AM

Bash uploader is being deprecated, though there has been no indication on when they are fully deprecating it. https://docs.codecov.com/docs/about-the-codecov-bash-uploader

gtdang commented 2 months ago

Closing because this was just used to debug workflows.