Cannot deploy on libvirtd when qemu is installed in user profile

[lheckemann]

While trying to deploy

{
  test = {
    deployment.targetEnv = "libvirtd";
    deployment.libvirtd.headless = true;
  };
}

I get (after a successful build):

nixops deploy -d test
test> starting...
test> error: Failed to create domain from /tmp/nixops-tmplJpJyx/test-domain.xml
test> error: internal error: Child process (LC_ALL=C PATH=/nix/store/j7hjvhc719mwvpq2041sfdw4i44q9b69-iptables-1.6.1/bin:/nix/store/smwf8xzcmikii2rk63217qhjinlb0mgv-iproute2-4.9.0/bin:/nix/store/2427iz938wyg1sdpr1q2blqxd4w0b4yy-pm-utils-1.4.1/bin:/nix/store/2q5avk6qbqxymcnn1fv7fv418rnvaclr-numad-0.5/bin:/nix/store/1b8zlahq3hmjk34aslbgxlj25nb01wk1-numactl-2.0.10/bin:/nix/store/7zgygl4lcwzicsfqfkalscy9cyj1lrlz-bridge-utils-1.5/bin:/nix/store/rnq7mbrqsy4f4yfi5fcaccjis8gmvl38-dmidecode-3.0/bin:/nix/store/jzs2pn35fsanb7d36920h93pmwgij7wd-dnsmasq-2.76/bin:/nix/store/4kjrb5h3fdw5abpdz7s90gxzqrb4w1l4-ebtables-2.0.10-4/bin:/nix/store/rr9ayf1x34dlx0ady9crlplxxhav6km6-qemu-x86-only-2.8.0/bin:/nix/store/4d6f8hg5gv20nsbq7b52qzn6bcs4fvlh-coreutils-8.26/bin:/nix/store/f3vl26f3n18khgq1kybnzvwjbm0r9grg-findutils-4.6.0/bin:/nix/store/28kkn4w4k4ay6aihbi1x1hpmicz3wq88-gnugrep-3.0/bin:/nix/store/0xwrn1p8fp8h3cynszpgbmhmydbzhns5-gnused-4.4/bin:/nix/store/5swjdn16ivaqp8x35df8w6n1h5miyv0a-systemd-232/bin:/nix/store/7zgygl4lcwzicsfqfkalscy9cyj1lrlz-bridge-util
test>
error: command ‘['virsh', '-c', 'qemu:///system', 'create', '/tmp/nixops-tmplJpJyx/test-domain.xml']’ failed on machine ‘test’ (exit code 1)

[NotBrianZach]

might check here: https://nixos.org/nixops/manual/#idm140737318345376 this might be a "dumb comment", but don't you need a "logical" (what I think the file you show is) and "physical" specification? Perhaps it would give you an error when trying to create the deployment if you didn't have one.

[lheckemann]

What I have there is effectively a physical specification. The logical specification is empty because I haven't added anything to it, so it'll just be a default nixos install.

Regardless, all deployments to libvirtd that I've tried fail with this same error message — this is the minimal case I reduced it to.

[NotBrianZach]

ah yes sorry, it has been a month or two since I messed with nixops. If you wish to continue debugging yourself, one way you could go about it is to look in this file: https://github.com/NixOS/nixops/blob/master/nixops/backends/libvirtd.py you could try recreating commands with virsh/qemu-img manually, or perhaps just adding "print" statements everywhere e.g.: self._logged_exec(["virsh", "-c", "qemu:///system", "create", dom_file]) one thing that might also be helpful for anyone else that reads this issue is, what version of nixops are you using (also perhaps what os?) I have successfully deployed libvirt with nixops before, though my version is somewhat old. though when I did "nix-env -qaP * | fgrep -i nixops" I got this: nixpkgs.nixops nixops-1.4 nixos.nixops nixops-1.4 nixpkgs.nixopsUnstable nixops-2016-11-23

and if nixops unstable is the latest unstable channel version of nixops (looks to be 1.4 if it's from november), I have used that one to deploy libvirt successfully, (my os is nixos, I follow the stable channel, which is currently 17.03) though not without problems when deploying more than one machine, see https://github.com/NixOS/nixops/issues/617 (I use a hack to work around this problem, (closed) pull request mentioned in issue)

[NotBrianZach]

the libvirt implementation has also had problems in the past with permissions and having to work around security features, so that would be something to be aware of as well.

[lheckemann]

Hah, removing qemu from my profile fixed it...