edx / edx-arch-experiments

A plugin to include applications under development by the architecture team at edx
GNU Affero General Public License v3.0
0 stars 3 forks source link

Replace codecov in openedx-events #528

Closed robrap closed 3 weeks ago

robrap commented 7 months ago

Acceptance Criteria:

Additional repos

Codecov upload is now simply failing in some repos and we've removed it as a required check (and made it an optional upload step) to unblock merges. In these repos, we should come back and review this decision after the replacement is done in one repo. The ticket should cover 1) switching to the replacement choice, and 2) re-enabling any coverage checks that have been made optional in the following repos:

Background Context:

I've seen the codecov upload issue in openedx-events, so we'll fix there as an example.

robrap commented 7 months ago

I guess this could have been an openedx-events issue, but oh well. We should inform the maintainers of the repo on the PR.

robrap commented 7 months ago

Diana posted this doc on the DEPR ticket: https://docs.openedx.org/en/latest/developers/how-tos/use-python-coverage-comment.html

nedbat commented 7 months ago

The "Basic Usage" in the action's readme shows a two-workflow setup. We often didn't need this because we were adding the action to private repo, so no external contributions. The two-workflow setup allows tests to be run by untrusted contributors, and then trigger a section action that has credentials allowing it to finish.

With reduced access and the increase of forks, more pull requests will need the two-workflow setup. If an external pull request is used with the one-workflow setup, unfortunately, the workflow and all the jobs in the workflow succeed, but the comment hasn't been made. These messages are in the action's step:

Notice: HTTP Request: GET https://api.github.com/user "HTTP/1.1 403 Forbidden"
Notice: HTTP Request: GET https://api.github.com/repos/openedx/openedx-events/issues/[30](https://github.com/openedx/openedx-events/actions/runs/7467415648/job/20320880894#step:8:32)0/comments "HTTP/1.1 200 OK"
Notice: Adding new comment
Notice: HTTP Request: POST https://api.github.com/repos/openedx/openedx-events/issues/300/comments "HTTP/1.1 403 Forbidden"
Notice: Cannot post comment. This is probably because this is an external PR, so it's expected. Ensure you have an additional `workflow_run` step configured as explained in the documentation (or alternatively, give up on PR comments for external PRs).
Notice: Ending action
robrap commented 7 months ago
  1. This is the PR that I think @nedbat was working on: https://github.com/openedx/openedx-events/pull/300
  2. I'm moving this back to No Status.
  3. I think we may need to reassess if this is still a quick win, and also who we can get to resolve this? Since we don't own openedx-events, maybe we can convert this to an openedx-events issue and see if they will resolve?
robrap commented 7 months ago

Status Update: We think that Ned's comment above about the need for two github actions is what needs to be worked on, and the linked PR that was closed just used the single action.

@feanil: Is this something that can be added to the Maintenance Working Group?

feanil commented 7 months ago

Yes, but I'm not sure how it will be prioritized at the moment.

robrap commented 7 months ago

@feanil: FYI: @timmc-edx will be looking into fixing https://github.com/edx/edx-arch-experiments/issues/437, which may be useful to the Maintenance WG for reviewing errors across repos.

jristau1984 commented 3 weeks ago

Closing due to inactivity and not owning this repo. The team has done what it can for the codecov issue, and now the community will complete the effort.