search

NixOS / ofborg

@ofborg tooling automation https://monitoring.ofborg.org/dashboard/db/ofborg
https://ofborg.org
MIT License
236 stars 165 forks source link

Build a lot more stuff #30

Open grahamc opened 6 years ago

grahamc commented 6 years ago

We've got a good bit of build capacity and it goes largely unused:

builds

Worse, x86_64-linux has effectively unbounded capacity due to a generous spot instance backing. Darwin is the least strong, and aarch64-linux is in the middle.

For these reasons, I'd like to build way more PRs and use this capacity to make merging PRs easier and safe.

Phase One: Simple Expansion

Phase Two: Easier Build Selection

Phase Three: Limited Automatic Building

Phase Four: Automatic Building

zimbatm commented 6 years ago

I asked @macstadium if they could provide some build capacity, no answer so far

7c6f434c commented 6 years ago

So, how many macOS builders there are now? Maybe (temporarily) allow to separately ask borg to build just i686-linux and x86_64-linux, to emphasize abundance of build capacity on this platform?

grahamc commented 6 years ago

Right now we have three mac builders. I think that is a good idea, @7c6f434c, and solves the issue around darwin not having adequate sandboxing for this use case. I am planning on the following:

Users bucketed by the following:

  1. if you are a trusted user, the build/test job is sent to all architectures.
  2. if you are a known user, the build/test job is only sent to architectures which has adequate sandboxing (aarch64-linux, x86_64-linux).
  3. if you are unknown, the command is ignored (as it is now)

With the addition of this change, all known members will be allowed to trigger the eval command.

7c6f434c commented 6 years ago

Erm. Doesn't i686-linux have adequate sandboxing? Aren't all its builders actually x86_64-linux?

Also, given the capacity skew, I think a trusted user might still want to opt out from sending to darwin?

grahamc commented 6 years ago

Let's wait and see if we actually have capacity issues on mac before scaling it back for trusted users. Ofborg doesn't run anything on i686.

LnL7 commented 6 years ago

I agree, we can monitor the queue and revisit if necessary. At the moment we actually have the same number of builders for every platform.

7c6f434c commented 6 years ago

Let's wait and see if we actually have capacity issues on mac before scaling it back for trusted users. Ofborg doesn't run anything on i686.

I guess what you meant is «before spending effort on cofigurability» (my proposal gives the trusted users an option to scale back, but allows them full runs, too). Fair enough, though.

grahamc commented 6 years ago

A simplified way to sample PRs. During the evaluation step:

If the user is trusted, send it to all platforms. If the user is known, send it to just platforms with good sandboxing. If the user is unknown, don't schedule any jobs.

This comes with the benefit of encouraging precisely following of the commit format. For example, knowing your package will auto-build may be the difference between:

grv: init at 0.1.0

and:

gitAndTools.grv: init at 0.1.0

andir commented 6 years ago

You might not always want to build it or at least have a blacklist? I think building chromium would waste a lot of cpu cycles with probably low benefit?

7c6f434c commented 6 years ago

This comes with the benefit of encouraging precisely following of the commit format.

gitAndTools.grv: init at 0.1.0

There are some fixes that simultaneously fix two or three packages and logically should be committed together; maybe support splitting by «,»?

7c6f434c commented 6 years ago

In the ideal world, Chromium would be built on the platforms with proper sandboxing, except those where the submitter has tried to build before submitting. (In the ideal world everyone uses the best sandboxing available for tests)

7c6f434c commented 6 years ago

By the way, judging from the acoustic evidence, there is significant progress on the original goal of builder utilisation. At least for some combinations of day-of-the-week and hour-of-the-day.

wmertens commented 6 years ago

How about building all dependencies for a given PR?

7c6f434c commented 6 years ago

@wmertens I think ofBorg currently doesn't allow to build just Chromium alone (timeout will happen sooner). What you say is way above capacity right now.

wmertens commented 6 years ago

aha ok :)

other question: how about running headless darwin in VMs? That way would solve Darwin capacity, right?

Another option might be cross-compiling but Nixpkgs is not there yet…

On Fri, Mar 23, 2018, 6:17 PM Michael Raskin, notifications@github.com wrote:

@wmertens https://github.com/wmertens I think ofBorg currently doesn't allow to build just Chromium alone (timeout will happen sooner). What you say is way above capacity right now.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/NixOS/ofborg/issues/30#issuecomment-375738511, or mute the thread https://github.com/notifications/unsubscribe-auth/AADWlsKPRs3vsGhZzAKokpdH5QdmbQnmks5thS4zgaJpZM4RMW4f .

7c6f434c commented 6 years ago

I don't think we have even x86_64-linux capacity.

Is there any Darwin option that can be legally ran on non-Apple hardware and is close enough to macOS for our build needs?

wmertens commented 6 years ago

hmm, legally :-/ there is https://github.com/PureDarwin/PureDarwin but it doesn't look like it actually works…

On Fri, Mar 23, 2018, 9:00 PM Michael Raskin, notifications@github.com wrote:

I don't think we have even x86_64-linux capacity.

Is there any Darwin option that can be legally ran on non-Apple hardware and is close enough to macOS for our build needs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NixOS/ofborg/issues/30#issuecomment-375782227, or mute the thread https://github.com/notifications/unsubscribe-auth/AADWlgSoJPAO96b9SMyI9LIRz_qQQZ-Jks5thVR2gaJpZM4RMW4f .

7c6f434c commented 6 years ago

It might even work per se, but Nixpkgs seems to depend on some Apple frameworks…

Anyway, we don't even have enough capacity for crazy megaprojects on the Linux side.

7c6f434c commented 5 years ago

I think some checkboxes here are either obsolete or done…