Meta github check

[zimbatm]

The list of github "checks" that garnix will set on a commit is variable. It depends on the shape of the flake output.

Problem statement

In the github branch protection setting, I can tell github to only merge a PR if certain checks are passing. And I can add the checks that Garnix set during the last run in there.

The only issue is that the list of Garnix checks evolves over time. As I add more packages, I will get more checks. And there is no way to tell GitHub to ask for "all the checks". So I have to go back in there and update the list as it changes.

Proposed solution

Have Garnix also publish a "all the garnix checks passed" check. Then I only have to add that one to the GitHub branch protection setting.

[Mic92]

Yeah. That is really needed. For example I have to manually update a long list of packages when using things like mergify: https://github.com/Mic92/sops-nix/blob/master/.mergify.yml

[Mic92]

An all the garnix checks passed would make my life a lot easier.

[jkarni]

I imagine changing the behavior of the Build starting build is also an option? I'm inclined to think that's slightly nicer UX, since then you have a single build that summarizes all others ("cloning" -> "evaluating builds" -> "building" -> "done")

[Mic92]

Hercules actually solves this nicely: https://github.com/nix-community/infra/pull/227

[Mic92]

If you have a lot of packages the list might become too long anyway.

[jkarni]

Hercules actually solves this nicely: https://github.com/nix-community/infra/pull/227 I'm a bit unclear on what's going on there - can you explain?

[Mic92]

It's a bit confusing because mergifio also added its status updates there.

But basically hercules ci add these three status updates that are updated in this order:

1. This is when all builds finished:

   • @hercules-ci ci/hercules/derivations — Success! Details

2. This is when evaluation is finished:

   • @hercules-ci ci/hercules/evaluation — 12 attributes Details

3. This is the impure bit of hercules ci that allows deploying (garnix.io does not have that)

   • @hercules-ci ci/hercules/effects — Job has no effects Details

[robx]

Screenshot for the previous comment (was a bit unclear where this shows):

[zimbatm]

Single status is probably good enough and would keep things simple. I can't think of a case where an evaluation check is interesting to wait on.

[sellout]

I would like something like this, too (not least because the GH interface for changing required statuses is painful.

It would be cool if the checks included in the meta-check could be configured via garnix.yaml, somehow indicating which builds are required, like

builds:
  include:
    - '*.aarch64-linux':
        required: true
    - '*.aarch64-linux.*'

But this could perhaps lead to issues where a malicious PR could disable required checks in the commit that are missed during review (but I think the meta-check already has this issue, which is why, IIUC, the GH required checks aren’t configurable in the code).

[sellout]

garnix now has an “All Garnix checks” check that I think satisfies this issue.