search

nix-community / infra

nix-community infrastructure [maintainer=@zowoq]
https://nix-community.org
MIT License
119 stars 81 forks source link

terraform/hydra-projects: add microvm.nix #1538

Closed astro closed 1 month ago

astro commented 1 month ago

I saw the NixCon lightning talk yesterday and it reminded me that I'd really love to have some test runners for microvm.nix.

I don't want to move the repo to this org though for stability and vanity reasons. Is that okay?

zowoq commented 1 month ago

test runners for microvm.nix.

I don't want to move the repo to this org though

This PR would only give you access to our public build box (12 core x86_64-linux).

We don't give out ssh access to our 48 core x86_64-linux, 80 core aarch64-linux machines. If you want access to our buildbot CI (for PR tests) the repo would need to be moved, otherwise we could give you a jobset on our hydra?

astro commented 1 month ago

A jobset on Hydra would be sufficient. The flakeref is github:astro/microvm.nix

zowoq commented 1 month ago

https://hydra.nix-community.org/jobset/microvm-nix/main

astro commented 1 month ago

Thank you very much!

zowoq commented 1 month ago

@astro My fault for not checking, I was expecting a flake with some VM tests (e.g. ~20) but it seems you have ~200.

Unfortunately our hydra is configured to best work with the cuda jobset, which means only 2 concurrent jobs on each machine, the amount of VM tests doesn't work well with that as some of them seem to have long timeouts or otherwise get stuck, blocking other jobs. I bumped concurrent jobs and restarted to try to get one one complete eval but I still ended up cancelling several of the vm tests.

The concurrent jobs problem is something is we're working on, I'm hoping that we'll make progress on that in the next few weeks but for now I've disabled the microvm jobset so it doesn't interfere with our existing jobsets.

zimbatm commented 4 weeks ago

The https://github.com/apps/buildbot-nix-community app could be installed on the repo instead.

zowoq commented 4 weeks ago

The https://github.com/apps/buildbot-nix-community app could be installed on the repo instead.

How can we run it as a public app without it being a security issue?

Mic92 commented 4 weeks ago

We could add an org whitelist feature to buildbot-nix.

zowoq commented 4 weeks ago

We could add an org whitelist feature to buildbot-nix.

That'll work, though I'd suggest scoping it to repos. (might also be able to use this instead of needing to set a topic?)

There is also https://github.com/nix-community/buildbot-nix/issues/176 which looks like it would be very similar to a hydra jobset.