nix-community / buildbot-nix

A nixos module to make buildbot a proper Nix-CI [maintainer=@Mic92,@MagicRB]
MIT License
114 stars 23 forks source link

postBuildStep's should probably be scheduled regardless of build skips #295

Open antifuchs opened 1 month ago

antifuchs commented 1 month ago

I have an "effect" (not a hercules-ci-effect! no idea how to use those tbh) via a postBuildStep that I would like to run only on the default branch of my repo. If I make a pull request that affects that attribute, it properly gets run on the PR branch and decides it has nothing to do; but if I merge the PR, the build is skipped and the postBuildStep isn't executed - it isn't even scheduled.

This is what a build on a PR branch looks like:

image

And this is on the merge commit:

image

I believe what happens here is that the evaluation counts as "cached" coming out of nix-eval-jobs, which prompts buildbot-nix to not build & then not schedule the following postBuildSteps either.

What I think would be neat is if we could:

Mic92 commented 1 month ago

I think these postBuildSteps should be just used for custom binary cache upload logic rather than continuous delivery and I rather not want to make building individual packages more expensive by adding more steps since it's already quite slow (We see this on flakes building thousands of packages). This would be than only available when we merged: https://github.com/nix-community/buildbot-nix/pull/68

Mic92 commented 1 month ago

If you want to make Hercules Ci happen faster, you can probably help us implementing it.

antifuchs commented 1 month ago

Yeah, I can see how abusing the "build" metaphor isn't something you would want to do. That said, I can't wrap my head around how hercules-ci effects works, and how it's supposed to fit into buildbot-nix. Is there a design doc I can look at?

Mic92 commented 1 month ago

We will probably write documentation/examples and use it our self once it is in. Otherwise I can just refer to hercules-ci own documentation.

antifuchs commented 1 month ago

I get that - but how will hci effects integrate into buildbot? Will a worker act as the equivalent of the hercules-ci agent, or will that still require the agent to be in place? Probably should take that discussion to #68 (:

Mic92 commented 1 month ago

I already implemented the runner interface of hercules-ci effects, so it will run in a buildbot worker as in a bubblewrap container.

Mic92 commented 1 month ago

https://github.com/nix-community/buildbot-nix/blob/3ac54ba1cc53aae01dd27055a710fbebab19acfe/buildbot_effects/__init__.py#L157

Mic92 commented 1 month ago

https://github.com/nix-community/buildbot-nix/pull/68/files#diff-df8c266d76f942a320d71b583a24da5fa8ecd8135993a696f376dbd960359be7R1264