nix-community / buildbot-nix

A nixos module to make buildbot a proper Nix-CI [maintainer=@Mic92,@MagicRB]
MIT License
120 stars 23 forks source link

postBuildStep's should probably be scheduled regardless of build skips #295

Open antifuchs opened 2 months ago

antifuchs commented 2 months ago

I have an "effect" (not a hercules-ci-effect! no idea how to use those tbh) via a postBuildStep that I would like to run only on the default branch of my repo. If I make a pull request that affects that attribute, it properly gets run on the PR branch and decides it has nothing to do; but if I merge the PR, the build is skipped and the postBuildStep isn't executed - it isn't even scheduled.

This is what a build on a PR branch looks like:

image

And this is on the merge commit:

image

I believe what happens here is that the evaluation counts as "cached" coming out of nix-eval-jobs, which prompts buildbot-nix to not build & then not schedule the following postBuildSteps either.

What I think would be neat is if we could:

Mic92 commented 2 months ago

I think these postBuildSteps should be just used for custom binary cache upload logic rather than continuous delivery and I rather not want to make building individual packages more expensive by adding more steps since it's already quite slow (We see this on flakes building thousands of packages). This would be than only available when we merged: https://github.com/nix-community/buildbot-nix/pull/68

Mic92 commented 2 months ago

If you want to make Hercules Ci happen faster, you can probably help us implementing it.

antifuchs commented 2 months ago

Yeah, I can see how abusing the "build" metaphor isn't something you would want to do. That said, I can't wrap my head around how hercules-ci effects works, and how it's supposed to fit into buildbot-nix. Is there a design doc I can look at?

Mic92 commented 2 months ago

We will probably write documentation/examples and use it our self once it is in. Otherwise I can just refer to hercules-ci own documentation.

antifuchs commented 2 months ago

I get that - but how will hci effects integrate into buildbot? Will a worker act as the equivalent of the hercules-ci agent, or will that still require the agent to be in place? Probably should take that discussion to #68 (:

Mic92 commented 2 months ago

I already implemented the runner interface of hercules-ci effects, so it will run in a buildbot worker as in a bubblewrap container.

Mic92 commented 2 months ago

https://github.com/nix-community/buildbot-nix/blob/3ac54ba1cc53aae01dd27055a710fbebab19acfe/buildbot_effects/__init__.py#L157

Mic92 commented 2 months ago

https://github.com/nix-community/buildbot-nix/pull/68/files#diff-df8c266d76f942a320d71b583a24da5fa8ecd8135993a696f376dbd960359be7R1264