Make PRs from external user repositories pass or skip CI jobs

teor2345 commented 2 years ago

Motivation

Currently, Zebra's CI fails on PRs from external contributors. This can be a really confusing and negative experience for a first-time contributor.

Example: (recent examples first) https://github.com/ZcashFoundation/zebra/pull/7516 https://github.com/ZcashFoundation/zebra/pull/4527

Jobs that need to be skipped

We don't want external PRs running on our Google Cloud, so we need to skip all those jobs. We want to skip release and deployment jobs as well, because they use secret keys.

See the full list here: https://github.com/ZcashFoundation/zebra/issues/4529#issuecomment-1396055038

Jobs that need to be fixed

All other jobs should be fixed if they don't work for external PRs.

Designs

We can skip some tests that would otherwise fail, because Mergify will catch them when it does a full test run. But we need to disable in-place merges to make sure Mergify does a separate run.

This is more secure, because we review code, approve it, then run Mergify on it.

Related Work

Our previous attempts to fix this issue didn't work:

3419

teor2345 commented 2 years ago

We're getting closer to the release candidate series, so this is a medium priority now.

teor2345 commented 1 year ago

Attracting external developers is not one of the goals of the release candidate, so this is a low priority.

mpguerra commented 1 year ago

Is it just as simple as skipping the CI Docker workflow or are there others that would need to be skipped?

How will this change affect the complexity of our CI rules and/or setup?

teor2345 commented 1 year ago

Is it just as simple as skipping the CI Docker workflow or are there others that would need to be skipped?

We'll also need to skip other workflows that write to our GitHub or Docker:

release drafter (PR auto-labels)
zcash-lightwalletd (google cloud Docker image push)

And avoid sending our GitHub access token to the protoc installer, by doing one of these things:

just skip those entire workflows (simplest option, and a tiny amount of work)
skip the protoc and --all-features parts of those workflows for external users (slightly more work)
commit the generated files to our git repository, and have a workflow that checks they are up to date (this would take slightly more work, but simplify a lot of other workflows, and decrease our CI failure

If we decide we really want one of these workflows to run, we can do the more complicated thing for it later.

How will this change affect the complexity of our CI rules and/or setup?

If we choose the simplest option, it is:

change the workflow triggers for about 5 workflows by adding 2 lines of conditions
change the patch workflows for those workflows to run instead

If we change the patch workflows, I don't think we even need to change Mergify at all. Which keeps things a lot simpler.

gustavovalverde commented 1 year ago

I'll be taking this into account for the redesign as we should really solve this for contributors.

mpguerra commented 1 year ago

Let's try to do this in 2023 Sprint 13 if we have time

teor2345 commented 1 year ago

This is a higher priority now we know QEDIT is going to start building on Zebra.

teor2345 commented 1 year ago

Jobs that need to be skipped

We don't want external PRs running on our Google Cloud, so we need to skip all those jobs. We want to skip release and deployment jobs as well, because they use secret keys.

Jobs that need to be fixed

All other jobs should be fixed if they don't work for external PRs.

gustavovalverde commented 12 months ago

While researching this (as I previously made changes to allow this behavior, by removing most secrets) I recently realized there's an open discussion as GitHub variables are impeding this from happening. And reverting from variables to fix values is a no-go: https://github.com/orgs/community/discussions/44322

In the meanwhile, the "best" approach is to request reviewers to trigger the actions after validating the proposed code, which would use the reviewer permissions.

I made this change as a workaround, and I'll be closing this ticket as not-planned for now.

teor2345 commented 12 months ago

What does this look like? Is there a test PR somewhere?

gustavovalverde commented 12 months ago

I've seen this in other open-source repos were I've contribute to. It's basically a button around here, asking for permissions to run the workflows

teor2345 commented 10 months ago

This fix does not work completely. Most CI runs successfully, but anything that uses Google Cloud doesn't work due to authentication issues. Let's test the next fix either before or after making it?

teor2345 commented 10 months ago

Here's a quick solution to this issue:

skip the jobs that are failing if a PR isn't from a branch in ZcashFoundation/zebra, and add patch jobs (or change their run conditions)
let mergify check those jobs once the PR is approved

ZcashFoundation / zebra