xen fails to build on unstable

[tfmoraes]

Describe the bug

Xen is not building this is the error:

builder for '/nix/store/m4id5qm2a742c11xzrx03ix1z29jwxps-xen-4.10.4.drv' failed with exit code 2; last 10 log lines:
  make[5]: Leaving directory '/build/xen-4.10.4/tools/firmware'
  make[4]: *** [Makefile:37: all] Error 2
  make[4]: Leaving directory '/build/xen-4.10.4/tools/firmware'
  make[3]: *** [/build/xen-4.10.4/tools/../tools/Rules.mk:245: subdir-install-firmware] Error 2
  make[3]: Leaving directory '/build/xen-4.10.4/tools'
  make[2]: *** [/build/xen-4.10.4/tools/../tools/Rules.mk:240: subdirs-install] Error 2
  make[2]: Leaving directory '/build/xen-4.10.4/tools'
  make[1]: *** [Makefile:74: install] Error 2
  make[1]: Leaving directory '/build/xen-4.10.4/tools'
  make: *** [Makefile:127: install-tools] Error 2

To Reproduce Steps to reproduce the behavior:

1. nix-shell -p xen
2. The error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

Notify maintainers

Metadata Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

• system: "x86_64-linux"
• host os: Linux 5.10.4, NixOS, 21.03.20210104.1a57d96 (Okapi)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.4pre20201201_5a6ddb3
• channels(root): "nixos-21.03pre250162.34ad166a830"
• nixpkgs: /etc/nixpkgs

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[andersk]

Actual error (from nix log nixpkgs.xen or Hydra):

drivers/infiniband/flexboot_nodnic.c: In function 'flexboot_nodnic_create_qp':
drivers/infiniband/flexboot_nodnic.c:368:53: error: implicit conversion from 'enum ib_queue_pair_type' to 'nodnic_queue_pair_type' [8;;https://gcc.gnu.org/on
linedocs/gcc/Warning-Options.html#index-Wenum-conversion^G-Werror=enum-conversion8;;^G]
  368 |  status = nodnic_port_create_qp(&port->port_priv, qp->type,
      |                                                   ~~^~~~~~
drivers/infiniband/flexboot_nodnic.c: In function 'flexboot_nodnic_destroy_qp':
drivers/infiniband/flexboot_nodnic.c:409:45: error: implicit conversion from 'enum ib_queue_pair_type' to 'nodnic_queue_pair_type' [8;;https://gcc.gnu.org/on
linedocs/gcc/Warning-Options.html#index-Wenum-conversion^G-Werror=enum-conversion8;;^G]
  409 |  nodnic_port_destroy_qp(&port->port_priv, qp->type,
      |                                           ~~^~~~~~

Possibly triggered by the GCC 10 upgrade.

[tfmoraes]

@andersk I found this patch https://lists.ipxe.org/pipermail/ipxe-devel/2020-May/007047.html

[tfmoraes]

Also, it seems to me that Xen is not needed in gnome-boxes. I created an overlay removing xen from buildInputs and it builds without a problem. Also I saw that there is no Xen as dependency in Gnome-boxes Flatpak neither in its Debian Package.

[bryanasdev000]

Also, it seems to me that Xen is not needed in gnome-boxes. I created an overlay removing xen from buildInputs and it builds without a problem. Also I saw that there is no Xen as dependency in Gnome-boxes Flatpak neither in its Debian Package.

Can confirm here, also I am bit confused too, in my case xen is failing to build for libguestfs which in turn is needed for vagrant, and I did not understand why, from a quick search Debian neither Arch uses xen as a dep in neither of this packages.

@andersk I found this patch https://lists.ipxe.org/pipermail/ipxe-devel/2020-May/007047.html

Will try.

EDIT: For ref, Gentoo team bug report: https://bugs.gentoo.org/722930 (Already solved)

[tfmoraes]

I submitted a PR to remove Xen as dependency from Gnome-boxes. https://github.com/NixOS/nixpkgs/pull/108716

[bryanasdev000]

Guys I am a bit lost here, I failed to apply the patches here, one motive is that is for 4.13+ and we are in 4.10, other is the needed options for building in GCC10, I do not know exactly where to put it, based on the patches from Gentoo (xen-tools-4.13.0-gcc10.patch and xen-tools-4.13.0-ipxe-gcc10.patch, available here https://github.com/hydrapolic/gentoo-dist/blob/master/xen/xen-gentoo-patches-21.tar.xz and here https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ab132ad91fa4e41eca3bc48eebae586c6ba58a11) the IPXE one, set the CFLAGS for the whole project, for example:

diff --git a/src/Makefile b/src/Makefile
index d74565d..2ad5b39 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -4,7 +4,7 @@
 #

 CLEANUP                :=
-CFLAGS         :=
+CFLAGS         := -Wno-enum-conversion -Wno-array-bounds -fcommon
 ASFLAGS                :=
 LDFLAGS                :=
 HOST_CFLAGS    :=

The other patch changes some little things plus a modifies a CFLAG for a specific module:

--- xen-4.13.0/tools/xenstore/utils.h.orig  2019-12-17 14:23:09.000000000 +0000
+++ xen-4.13.0/tools/xenstore/utils.h   2020-01-21 21:13:05.108957447 +0000
@@ -24,7 +24,7 @@
 void barf(const char *fmt, ...) __attribute__((noreturn));
 void barf_perror(const char *fmt, ...) __attribute__((noreturn));

-void (*xprintf)(const char *fmt, ...);
+extern void (*xprintf)(const char *fmt, ...);

 #define eprintf(_fmt, _args...) xprintf("[ERR] %s" _fmt, __FUNCTION__, ##_args)

--- xen-4.13.0/tools/xenstore/xenstored_core.h.orig 2020-01-21 21:15:19.243931307 +0000
+++ xen-4.13.0/tools/xenstore/xenstored_core.h  2020-01-21 21:38:35.340617819 +0000
@@ -204,7 +204,7 @@
 /* Open a pipe for signal handling */
 void init_pipe(int reopen_log_pipe[2]);

-xengnttab_handle **xgt_handle;
+extern xengnttab_handle **xgt_handle;

 int remember_string(struct hashtable *hash, const char *str);

--- xen-4.13.0/tools/libxl/libxlu_pci.c.orig    2019-12-17 14:23:09.000000000 +0000
+++ xen-4.13.0/tools/libxl/libxlu_pci.c 2020-01-21 21:56:26.812212916 +0000
@@ -48,7 +48,7 @@
 int xlu_pci_parse_bdf(XLU_Config *cfg, libxl_device_pci *pcidev, const char *str)
 {
     unsigned state = STATE_DOMAIN;
-    unsigned dom, bus, dev, func, vslot = 0;
+    unsigned dom = 0, bus = 0, dev = 0, func = 0, vslot = 0;
     char *buf2, *tok, *ptr, *end, *optkey = NULL;

     if ( NULL == (buf2 = ptr = strdup(str)) )
--- xen-4.13.0/tools/libxl/libxl_utils.c.orig   2019-12-17 14:23:09.000000000 +0000
+++ xen-4.13.0/tools/libxl/libxl_utils.c    2020-01-21 22:34:52.096300774 +0000
@@ -1259,7 +1259,7 @@
     }
     memset(un, 0, sizeof(struct sockaddr_un));
     un->sun_family = AF_UNIX;
-    strncpy(un->sun_path, path, sizeof(un->sun_path));
+    strncpy(un->sun_path, path, sizeof(un->sun_path)-1);
     return 0;
 }

--- xen-4.13.0/tools/debugger/kdd/kdd.h.orig    2019-12-17 14:23:09.000000000 +0000
+++ xen-4.13.0/tools/debugger/kdd/kdd.h 2020-01-21 23:35:55.458605582 +0000
@@ -323,7 +323,7 @@
         kdd_msg msg;
         kdd_reg reg;
         kdd_stc stc;
-        uint8_t payload[0];
+        uint8_t payload[65535];
     };
 } PACKED kdd_pkt;

--- xen-4.13.0/tools/xenpmd/Makefile.orig   2019-12-17 14:23:09.000000000 +0000
+++ xen-4.13.0/tools/xenpmd/Makefile    2020-01-22 22:13:16.564873608 +0000
@@ -3,6 +3,7 @@

 CFLAGS += -Werror
 CFLAGS += $(CFLAGS_libxenstore)
+CFLAGS += -Wno-error=format-truncation

 LDLIBS += $(LDLIBS_libxenstore)

But there is no definition for this in our makefile (source https://github.com/xen-project/xen/blob/stable-4.10/Makefile).

I did try to add it on NIX_CFLAGS_COMPILE as 50dad9e873d5ee60f008ab0d415afde7b4f3acdd did

But adding --fcommon gives error on the configuration step:

configure: running /nix/store/zcl19h06322c3kss6bvf05w2pxg4kfll-bash-4.4-p23/bin/bash ./configure --disable-option-checking '--prefix=/nix/store/9ajj3drb9aqgin2qjqnyiab54x40h52d-xen-4.10.4'  '--enable-qemu-traditional' --cache-file=/dev/null --srcdir=.
configure: WARNING: Setting CC, CFLAGS, LDFLAGS, LIBS, CPPFLAGS or CPP is not recommended, use PREPEND_INCLUDES, PREPEND_LIB, APPEND_INCLUDES and APPEND_LIB instead when possible.
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/build/xen-4.10.4/tools':
configure: error: C compiler cannot create executables
See `config.log' for more details
configure: error: ./configure failed for tools
builder for '/nix/store/b4wv1j8wmn2d4x35p6rp4s19v0g6qjiq-xen-4.10.4.drv' failed with exit code 77
cannot build derivation '/nix/store/xh360fdfvisvvxb5wj7nzbismdrlnli7-libguestfs-1.40.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/n319dgfqmv0vmn3322wrj731yqasvc9r-vagrant-2.2.14.drv': 1 dependencies couldn't be built

Without it, advances a lit bit more but also fails:

cc1plus: error: command-line option '-Wno-enum-conversion' is valid for C/ObjC but not for C++ [-Werror]
cc1plus: all warnings being treated as errors
make[10]: *** [Makefile:123: headers++.chk] Error 1
rm compat/callback.c compat/memory.c compat/arch-x86/xen-mca.c compat/tmem.c compat/arch-x86/xen-x86_32.c compat/nmi.c compat/hvm/hvm_vcpu.c compat/grant_table.c compat/physdev.c compat/version.c compat/hvm/dm_op.c compat/sched.c compat/trace.c compat/xenoprof.c compat/features.c compat/arch-x86_32.c compat/kexec.c compat/platform.c compat/arch-x86/pmu.c compat/arch-x86/xen.c compat/pmu.c compat/xen.c compat/vcpu.c
make[10]: Leaving directory '/build/xen-4.10.4/tools/firmware/xen-dir/xen-root/xen/include'
make[9]: *** [Makefile:142: /build/xen-4.10.4/tools/firmware/xen-dir/xen-root/xen/xen] Error 2
make[9]: Leaving directory '/build/xen-4.10.4/tools/firmware/xen-dir/xen-root/xen'
make[8]: *** [Makefile:45: build] Error 2
make[8]: Leaving directory '/build/xen-4.10.4/tools/firmware/xen-dir/xen-root/xen'
make[7]: *** [Makefile:51: xen-shim] Error 2
make[7]: Leaving directory '/build/xen-4.10.4/tools/firmware/xen-dir'
make[6]: *** [/build/xen-4.10.4/tools/firmware/../../tools/Rules.mk:245: subdir-all-xen-dir] Error 2
make[6]: Leaving directory '/build/xen-4.10.4/tools/firmware'
make[5]: *** [/build/xen-4.10.4/tools/firmware/../../tools/Rules.mk:240: subdirs-all] Error 2
make[5]: Leaving directory '/build/xen-4.10.4/tools/firmware'
make[4]: *** [Makefile:37: all] Error 2
make[4]: Leaving directory '/build/xen-4.10.4/tools/firmware'
make[3]: *** [/build/xen-4.10.4/tools/../tools/Rules.mk:245: subdir-install-firmware] Error 2
make[3]: Leaving directory '/build/xen-4.10.4/tools'
make[2]: *** [/build/xen-4.10.4/tools/../tools/Rules.mk:240: subdirs-install] Error 2
make[2]: Leaving directory '/build/xen-4.10.4/tools'
make[1]: *** [Makefile:74: install] Error 2
make[1]: Leaving directory '/build/xen-4.10.4/tools'
make: *** [Makefile:127: install-tools] Error 2
builder for '/nix/store/vrsgfvq6sq692c8l19xn9zgaj4c1lxvv-xen-4.10.4.drv' failed with exit code 2
cannot build derivation '/nix/store/b8f0mhll77v62hcfgkkpxm1c8kb9l7j9-libguestfs-1.40.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/15g9m9x2r5cmrvjya4xx51s1m67mxvh0-vagrant-2.2.14.drv': 1 dependencies couldn't be built

From a bit more of research, Vagrant depends on libguestfs because of the libvirt plugin, which in turn depends on xen, I think is better to version bump our xen then patches it for GCC10. 

For now I using (vagrant.override { withLibvirt = false; }) on home-manager.users.bryan.home.packages. At least I can use Virtualbox for quick experiments.

[andersk]

If version bumping Xen fixes this, I can’t imagine a reason that wouldn’t be our first choice—we should do it.

[bryanasdev000]

If version bumping Xen fixes this, I can’t imagine a reason that wouldn’t be our first choice—we should do it.

It will help, but we will still need patches for GCC10 until we have a release to address that (from upstream). At least that's what I understood by reading the issues and PR from the Gentoo team.

[bryanasdev000]

Quick update, libguestfs doesn't need xen as a dependency, I'm trying to compile it without xen to allow vagrant and friends to build it again.

EDIT: Reference https://libguestfs.org/guestfs-building.1.html

[bryanasdev000]

Ok, the bump of autoconf to 2.70 also breaks building libguestfs on unstable, on 20.09 I can build libguestfs without xen and any problem.

[tfmoraes]

@bryanasdev000 I saw that autoconf needs this patch https://git.savannah.gnu.org/cgit/autoconf.git/commit/?h=branch-2.70&id=35b1708c3961dce31e4ce43014e71d77aa4b5e31

[tfmoraes]

@bryanasdev000 I submitted a PR to remove Xen from libguestfs: https://github.com/NixOS/nixpkgs/pull/108882

I needed to change autoconf to version 2.64.

[bryanasdev000]

@bryanasdev000 I submitted a PR to remove Xen from libguestfs: #108882

I needed to change autoconf to version 2.64.

I was just trying to force the build with an older version of autoconf, I tried to update libguestfs to version 1.44, it reduced the errors but it still did not build.

Thanks! Excellent job!

[bryanasdev000]

Guys with the fix of Vagrant (Libguestfs) I did not check Xen anymore, did anyone advance with that?

If not I can try to mess with it, but somebody will need to test since I don't use Xen at all.