NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
18.04k stars 14.04k forks source link

Add documentation for emulators feature in cross infra #106375

Open siraben opened 3 years ago

siraben commented 3 years ago

While working on cross-related PRs I came across emulators introduced by https://github.com/NixOS/nixpkgs/commit/9c8fd412248ad907eee7547b19bf3f7583d2c411. They seem useful but haven't been documented anywhere. IMO would be a good addition to the cross-compilation section of the manual once #105364 is merged.

As I understand it, some use cases include

Pinging (author of emulators feature) @matthewbauer, (cross infra maintainer) @Ericson2314

siraben commented 3 years ago

@matthewbauer could you describe a bit how emulators works, how to use it, and how to run the nixpkgs tests?

matthewbauer commented 3 years ago

The emulators property of {build,host,target}Platform provides a way to run binaries for one architecture on another. It selects from a package set some emulator that's known to provide this in Nixpkgs. For example:

# stdenv.hostPlatform.emulator :: PackageSet -> String
$ nix eval --impure --expr '(import <nixpkgs> {}).pkgsCross.aarch64-multiplatform.stdenv.hostPlatform.emulator (import <nixpkgs> {})'
"/nix/store/vfzcwwp4xlgzkfak0n422hgfr4kfdngq-qemu-5.1.0/bin/qemu-aarch64"

Emulators includes Wine, Qemu-user, & wasmtime, but the idea is to avoid specifying exactly which architecture you need.

More examples:

$ nix eval --impure --expr '(import <nixpkgs> {}).stdenv.hostPlatform.emulator (import <nixpkgs> {})'
"/nix/store/a3fc4zqaiak11jks9zd579mz5v0li8bg-bash-4.4-p23/bin/bash -c '\"$@\"' --"
$ nix eval --impure --expr '(import <nixpkgs> {}).pkgsCross.mingw32.stdenv.hostPlatform.emulator (import <nixpkgs> {})'
"/nix/store/g9qvz1y25d500h8fww9vdfmmyknxnd9c-wine-5.0/bin/wine32"
siraben commented 3 years ago

I see that there are emulator tests in Nixpkgs, how would one run them?

siraben commented 3 years ago

I tried a simple expression, but /nix/store/lzlk7kn4aw34r9m9hf04qca31s3s8q8l-wine-5.0.3/bin/wine32 doesn't exist.

{ pkgs ? import <nixpkgs> { }}:
with pkgs;

let
  mingw-hello = pkgsCross.mingw32.hello;
  emul = pkgsCross.mingw32.stdenv.hostPlatform.emulator pkgs;

in
rec {
  runit = writeScript "runit" ''
    #!/usr/bin/env sh
    ${emul} ${mingw-hello}/bin/hello
  '';
}

This worked, however

{ pkgs ? import <nixpkgs> { }}:
with pkgs;

let
  hello-cross = pkgsCross.aarch64-multiplatform.hello;
  emul = pkgsCross.aarch64-multiplatform.stdenv.hostPlatform.emulator pkgs;

in
rec {
  runit = writeScript "runit" ''
    #!/usr/bin/env sh
    ${emul} ${hello-cross}/bin/hello
  '';
}
siraben commented 3 years ago

@matthewbauer what kinds of things would this allow in Nixpkgs? There's several instances where packages use their own build artifacts as part of the build process, which wouldn't work when cross-compiling, so could we patch them to use the emulator to complete cross-compilation?

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

siraben commented 3 years ago

Still WIP

nixos-discourse commented 3 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/call-for-participation-nix-workshop-at-vandyhacks/15401/4

ghost commented 2 years ago

I think there are still two pieces missing here, but once they are added this will be quite a powerful feature:

  1. Every hostPlatform with a kernel in nixpkgs (i.e. at minimum, Linux) should know the nix expression that builds a kernel that works in the platform's emulator (e.g. qemu) without any extra fiddly configuration effort.

  2. Add an attribute emulatedBuild which is like pkgsCross but instead of changing only the hostPlatform to create a cross-compiled package it instead changes both the buildPlatform and the hostPlatform to create a native-compiled package, with all derivations wrapped in an emulator invocation.

I implemented (1) for one specific case (mips64el) here. The result is pretty nifty: you can use 9pfs to mount the host's /nix/store (read-only, of course) within the guest, so you don't even need a root filesystem image. Just a kernel, initramfs-with-busybox, and then you're off and away. Adding (2) would not be so hard.

This will be really handy for cross compilation situations where you have a long dependency chain of packages and all but one of them (buried somewhere in the middle of the chain) is able to cross-compile. For example, #166199. It would mean that an overlay could be used to swap out { gobject-introspection = emulatedBuild.gobject-introspection; }.

nixos-discourse commented 11 months ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/stdenv-hostplatform-emulator-use-case/35367/1