robur-coop / albatross

Albatross: orchestrate and manage MirageOS unikernels with Solo5
ISC License
141 stars 17 forks source link

Deploy albatross using NixOS #120

Closed Julow closed 1 year ago

Julow commented 1 year ago

The flake.nix file define a package for albatross as well as a nixos module for deploying it as systemd services.

The project is built with opam-nix, which should ensure that the package definition don't get out of date (as long as opam install works).

The systemd units are taken from the existing packaging/linux with slight changes. The module also configures a NAT and a network bridge named service.

The TLS endpoint is placed in a separate module to make sure it is optional.

An example of use can be found here: https://github.com/Julow/albatross-nixos-example

A weakness of the module at the moment is that the TLS endpoint's private key is copied into the Nix store and available to all users.

hannesm commented 1 year ago

sounds good to me, any chance we can have an github action that uses nix?

Julow commented 1 year ago

I finally manage to write an action that runs in a reasonable time (3 min). It could run much faster, it's still downloading a lot on every builds and even building Opam for some reason. This seem to be due to opam-nix's multi-step builds and this trick to reduce the size of the closure that both make many of the build dependencies unrooted (and garbage collected). There's room for improvement.

hannesm commented 1 year ago

Thanks for your work @Julow. I'm wondering whether -- similar to https://github.com/mirage/ocaml-git/pull/600 -- we can avoid to hardcode all the nix checksums, which is a burden on maintenance that I'd like to avoid. Does this sound feasible and reasonable to you?

Julow commented 1 year ago

The lock file only locks the version of Opam repositories, build tools and nixpkgs. All of which needs to be defined at some point, none of them are defined in nixpkgs.

The approach used in https://github.com/mirage/ocaml-git/pull/600 is not less work and maintenance in the case of albatross because it has a lot of dependencies that are not in nixpkgs. However, the advantage is that we wouldn't need opam-nix anymore (slow compilation, no dev shell) and no need to update the lock file once in a while.

Julow commented 1 year ago

I can give a try but this might take a bit of time.

hannesm commented 1 year ago

we can also merge this as is, but I won't be able to update the lock files on my own. If the nixos github action fails, I'd mention you so you can take a look - does this sound ok? (we can of course at a later point revise the github action)

Julow commented 1 year ago

I volunteer for maintaining the flake and the nixos module.

Et7f3 commented 1 year ago

The approach used in mirage/ocaml-git#600 is not less work and maintenance in the case of albatross because it has a lot of dependencies that are not in nixpkgs.

Have you a list of dependencies missings ?

Julow commented 1 year ago

Not that many actually! It seems that I gave up too soon:

[ "dune" "conf-pkg-config" "conf-libnl3" "metrics-rusage" "http-lwt-client" "happy-eyeballs-lwt" "solo5-elftool" ]

The first 3 are just noise, the rest should be reasonable to add, with not many transitive deps.

Et7f3 commented 1 year ago

Does https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/solo5/default.nix contains solo5-elftool ? Seem to be metrics-rusage https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/ocaml-modules/metrics/rusage.nix happy-eyeballs is here https://github.com/NixOS/nixpkgs/blob/dceb18596b0f12d874865e2ffd648794221d3fed/pkgs/top-level/ocaml-packages.nix#L526 I think http-lwt-client isn't packaged yet (need to double check)

Julow commented 1 year ago

solo5-elftool is https://git.robur.io/robur/ocaml-solo5-elftool. You right for the others!

hannesm commented 1 year ago

FWIW the solo5-elftool is mirrored (gitea push mirror) on GitHub https://github.com/roburio/ocaml-solo5-elftool (if this makes packaging easier in nixos, feel free to use that)