Cross compilation

[roberth]

Something like #82 but a bit more involved.

Interface ideas

perSystem = { getHostSystem, getBuildSystem, ... }: to set either system. top@{ getCrossSystem, ... }:, getCrossSystem { /* both host and build */ /* -platform or -system? :/ */ }

Do we set the perSystem to host? I think so, because the goal is to declare outputs, not program a build process. Otherwise we get something like crossSystem which doesn't have a simple, useful definition for reading.

getHostSystem memoization may be tricky. (Use tries??)

Require al host systems to be declared? Required for a name suffixing feature.

[woile]

I've been investigating this a bit, I found some potential approaches but I need some feedback. For now, my goal is to be able to build docker containers from Linux and Mac. Meaning that the cross-compilation that I'd like is for Mac to Linux. But depending on the tool used more things could be done. This could be an initial module, which would make a lot of us happy 🙌🏻

I think (if my understanding is correct), we could use nix "builders" for this. Meaning that you could just do something like: nix build .#dockerImage.aarch64-linux, and if the module detects it needs to cross-compile, spins the vm and it just works. I found darwin.builder in the docs, but looking at the flake, I have no idea how to use it. Do you think something like that could work?

In this post, it makes use of nix-darwin as well but with qemu, I'm still not sure how to derive a flake from that or darwin.builder. They all seem to require some modification of the host OS. Is it something we can expect from users?

Some other possible tools:

• lima
• colima (containers + lima)

I've considered nixpacks, but it actually doesn't seem to integrate well with nix, it does a lot of work that dream2nix already does and the images end up being quite heavy. It could be a potential initial step as well anyways, like a separate module nixpacks-flake.

---

Do we set the perSystem to host?

This is also a good approach (I don't know what would be best), do you envision something like this?

{
  perSystem = { config, self', inputs', pkgs, system, ... }: {
    packages.appDockerImage = config.crossCompile.packages.appDockerImage
  };
  crossCompile = {config, pkgs, ...}: {
    supportedSystems = ["x86_64-linux", "x86_64-darwin"]
    packages."x86_64-linux".app = dream2nix ./.;
    packages."x86_64-linux".appDockerImage = dockerTools.streamLayeredImage {...};
  }
}

[roberth]

Whatever we do is going to be not super nice because the Nix CLI only supports one "system" whereas we have two.

I've said before, all of open source is really just nixpkgs fan fiction, and I think that applies here rather well. perSystem tends to be just a likeNixpkgs function, as the system parameter isn't used directly anyway, and you can just replace pkgs by something useful and get a nice result. That's what easyOverlay does, converting perSystem to an overlay, and I think we can do something similar for cross compilation.

So back to the Nix CLI, two is not a lot more than one, and we users can manage that. The "tricky" bit is to keep track of how Nix is going to interpret each one, or in other words how the extra system relates to the currentSystem that Nix uses in its technically impure installable resolution logic. If that sounded vague, that's kind of the point. We shouldn't do that. I think we should always make clear whether we're compiling "to" or "from" the system that's explicit in the command line. Or maybe better, use "build" and "host"?

So we could both nix build .#to-aarch64-linux.default and nix build .#from-x86_64-linux.default. The latter being useful when we want to do nix run with a remote build on a machine that's a different platform.

As for how to declare it, perSystem should be enough, except we need to declare an extra list of systems or system pairs. Repetition of packaging logic should not be necessary.

[woile]

You raise very valid points. Specially with using 1 or 2 commands, I think it would be completely acceptable.

IIUC you are saying the system section should support "more" systems? Like when you do nix flake show, you'd see listed in the packages not only packages.x86_64-linux but also things like packages.from-x86_64-linux? How would something like that work? (I'm still trying to get to grip with all the nix ecosystem 😅 )

OTOH, do you see an issue with providing and documenting an easy way to set up remote builders? Wouldn't it then be a matter of starting the builder, and then running the nix build --builders 'ssh://a-linux x86_64-linux'?