Add documentation for emulators feature in cross infra

[siraben]

While working on cross-related PRs I came across emulators introduced by https://github.com/NixOS/nixpkgs/commit/9c8fd412248ad907eee7547b19bf3f7583d2c411. They seem useful but haven't been documented anywhere. IMO would be a good addition to the cross-compilation section of the manual once #105364 is merged.

As I understand it, some use cases include

• testing cross-built binaries
• allowing cross-building of some packages (especially some compilers and tools) that use cross-compiled binaries at build time

Pinging (author of emulators feature) @matthewbauer, (cross infra maintainer) @Ericson2314

[siraben]

@matthewbauer could you describe a bit how emulators works, how to use it, and how to run the nixpkgs tests?

[matthewbauer]

The emulators property of {build,host,target}Platform provides a way to run binaries for one architecture on another. It selects from a package set some emulator that's known to provide this in Nixpkgs. For example:

# stdenv.hostPlatform.emulator :: PackageSet -> String
$ nix eval --impure --expr '(import <nixpkgs> {}).pkgsCross.aarch64-multiplatform.stdenv.hostPlatform.emulator (import <nixpkgs> {})'
"/nix/store/vfzcwwp4xlgzkfak0n422hgfr4kfdngq-qemu-5.1.0/bin/qemu-aarch64"

Emulators includes Wine, Qemu-user, & wasmtime, but the idea is to avoid specifying exactly which architecture you need.

More examples:

$ nix eval --impure --expr '(import <nixpkgs> {}).stdenv.hostPlatform.emulator (import <nixpkgs> {})'
"/nix/store/a3fc4zqaiak11jks9zd579mz5v0li8bg-bash-4.4-p23/bin/bash -c '\"$@\"' --"

$ nix eval --impure --expr '(import <nixpkgs> {}).pkgsCross.mingw32.stdenv.hostPlatform.emulator (import <nixpkgs> {})'
"/nix/store/g9qvz1y25d500h8fww9vdfmmyknxnd9c-wine-5.0/bin/wine32"

[siraben]

I see that there are emulator tests in Nixpkgs, how would one run them?

[siraben]

I tried a simple expression, but /nix/store/lzlk7kn4aw34r9m9hf04qca31s3s8q8l-wine-5.0.3/bin/wine32 doesn't exist.

{ pkgs ? import <nixpkgs> { }}:
with pkgs;

let
  mingw-hello = pkgsCross.mingw32.hello;
  emul = pkgsCross.mingw32.stdenv.hostPlatform.emulator pkgs;

in
rec {
  runit = writeScript "runit" ''
    #!/usr/bin/env sh
    ${emul} ${mingw-hello}/bin/hello
  '';
}

This worked, however

{ pkgs ? import <nixpkgs> { }}:
with pkgs;

let
  hello-cross = pkgsCross.aarch64-multiplatform.hello;
  emul = pkgsCross.aarch64-multiplatform.stdenv.hostPlatform.emulator pkgs;

in
rec {
  runit = writeScript "runit" ''
    #!/usr/bin/env sh
    ${emul} ${hello-cross}/bin/hello
  '';
}

[siraben]

@matthewbauer what kinds of things would this allow in Nixpkgs? There's several instances where packages use their own build artifacts as part of the build process, which wouldn't work when cross-compiling, so could we patch them to use the emulator to complete cross-compilation?

[stale[bot]]

I marked this as stale due to inactivity. → More info

[siraben]

Still WIP

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/call-for-participation-nix-workshop-at-vandyhacks/15401/4

[ghost]

I think there are still two pieces missing here, but once they are added this will be quite a powerful feature:

1. Every hostPlatform with a kernel in nixpkgs (i.e. at minimum, Linux) should know the nix expression that builds a kernel that works in the platform's emulator (e.g. qemu) without any extra fiddly configuration effort.

2. Add an attribute emulatedBuild which is like pkgsCross but instead of changing only the hostPlatform to create a cross-compiled package it instead changes both the buildPlatform and the hostPlatform to create a native-compiled package, with all derivations wrapped in an emulator invocation.

I implemented (1) for one specific case (mips64el) here. The result is pretty nifty: you can use 9pfs to mount the host's /nix/store (read-only, of course) within the guest, so you don't even need a root filesystem image. Just a kernel, initramfs-with-busybox, and then you're off and away. Adding (2) would not be so hard.

This will be really handy for cross compilation situations where you have a long dependency chain of packages and all but one of them (buried somewhere in the middle of the chain) is able to cross-compile. For example, #166199. It would mean that an overlay could be used to swap out { gobject-introspection = emulatedBuild.gobject-introspection; }.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/stdenv-hostplatform-emulator-use-case/35367/1