search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.48k stars 13.67k forks source link

xorg packages don't cross compile #51178

Closed eburimu closed 5 years ago

eburimu commented 5 years ago

Issue description

More bad news I'm afraid.

 9287 [void@fermat:~/pi]$ sh ./xcb-proto.sh
 9288 these derivations will be built:
 9289   /nix/store/6z4fsn4ihi0vpsjaj5hyvgqp3vwml5vf-xcb-proto-1.13-armv7l-unknown-linux-gnueabihf.drv
 9290 building '/nix/store/6z4fsn4ihi0vpsjaj5hyvgqp3vwml5vf-xcb-proto-1.13-armv7l-unknown-linux-gnueabihf.drv'...
 9291 unpacking sources
 9292 unpacking source archive /nix/store/hrjhqp1b5zkvm6g6vqcadgs0dbzfjfdn-xcb-proto-1.13.tar.bz2
 9293 source root is xcb-proto-1.13
 9294 setting SOURCE_DATE_EPOCH to timestamp 1520241497 of file xcb-proto-1.13/xcbgen/Makefile.in
 9295 patching sources
 9296 updateAutotoolsGnuConfigScriptsPhase
 9297 configuring
 9298 configure flags: --prefix=/nix/store/zbabb6l8jsqjyxd67psbgk1xmc459z8i-xcb-proto-1.13-armv7l-unknown-linux-gnueabihf --build=x86_64-unknown-linux-gnu --host=armv7l-unknown-linux-gnueabihf
 9299 checking for a BSD-compatible install... /nix/store/9dajacm6s12cgqkc15zrrqgh26a1pd83-coreutils-8.30/bin/install -c
 9300 checking whether build environment is sane... yes
 9301 checking for armv7l-unknown-linux-gnueabihf-strip... armv7l-unknown-linux-gnueabihf-strip
 9302 checking for a thread-safe mkdir -p... /nix/store/9dajacm6s12cgqkc15zrrqgh26a1pd83-coreutils-8.30/bin/mkdir -p
 9303 checking for gawk... gawk
 9304 checking whether make sets $(MAKE)... yes
 9305 checking whether make supports nested variables... yes
 9306 checking for xmllint... no
 9307 configure: WARNING: xmllint not found; unable to validate against schema.
 9308 checking for a Python interpreter with version >= 2.5... none
 9309 configure: error: no suitable Python interpreter found
 9310 builder for '/nix/store/6z4fsn4ihi0vpsjaj5hyvgqp3vwml5vf-xcb-proto-1.13-armv7l-unknown-linux-gnueabihf.drv' failed with exit code 1
 9311 error: build of '/nix/store/6z4fsn4ihi0vpsjaj5hyvgqp3vwml5vf-xcb-proto-1.13-armv7l-unknown-linux-gnueabihf.drv' failed

Steps to reproduce

 9283 [void@fermat:~/pi]$ cat ./xcb-proto.sh
 9284 #!/bin/sh
 9285 nix-build -I nixpkgs=/home/void/nixpkgs -E 'let t1 = import <nixpkgs> {}; in let pkgs = import <nixpkgs> { crossSystem = t1.lib.systems.examples.armv7l-hf-multiplatform // { system = "armv7l-linux"; }; }; in pkgs.xorg.xcbproto ' --show-trace

Technical details

The code below in top-level/all-packages.nix doesn't work for cross compilation.

 13934 
 13935   # Use `lib.callPackageWith __splicedPackages` rather than plain `callPackage`                                                                                                                                                         
 13936   # so as not to have the newly bound xorg items already in scope,  which would                                                                                                                                                         
 13937   # have created a cycle.                                                                                                                                                                                                               
 13938   xorg = recurseIntoAttrs ((lib.callPackageWith __splicedPackages ../servers/x11/xorg {
 13939   }).overrideScope' (lib.callPackageWith __splicedPackages ../servers/x11/xorg/overrides.nix {
 13940     inherit (darwin) cf-private;
 13941     inherit (darwin.apple_sdk.frameworks) ApplicationServices Carbon Cocoa;
 13942     inherit (darwin.apple_sdk.libs) Xplugin;
 13943     bootstrap_cmds = if stdenv.isDarwin then darwin.bootstrap_cmds else null;
 13944     python = python2; # Incompatible with Python 3x                                                                                                                                                                                     
 13945     udev = if stdenv.isLinux then udev else null;
 13946     libdrm = if stdenv.isLinux then libdrm else null;
 13947     abiCompat = config.xorg.abiCompat # `config` because we have no `xorg.override`                                                                                                                                                     
 13948       or (if stdenv.isDarwin then "1.18" else null); # 1.19 needs fixing on Darwin                                                                                                                                                      
 13949   }) // { inherit xlibsWrapper; } );
 13950

Basically, the problem can be stated as follows. In pkgs/servers/x11/xorg/override.nix, xcbproto has the following derivation:

  94   xcbproto = super.xcbproto.overrideAttrs (attrs: {                                                                                                                                                                                       
  95     nativeBuildInputs = attrs.nativeBuildInputs ++ [ python ];                                                                                                                                                              
  96   });

Unfortunately, python in this nat. buil. input turns out to be the cross one. The following fixes the problem:

  94   xcbproto = super.xcbproto.overrideAttrs (attrs: {                                                                                                                                                                                       
  95     nativeBuildInputs = attrs.nativeBuildInputs ++ [ buildPackages.python ];                                                                                                                                                              
  96   });

Obviously, this has to be done all packages w/ nat. bui. inp.'s. Maybe it is possible to fix all-packages.nix, any ideas?

globin commented 5 years ago

I have a fix for this I'm just about to PR, just have to merge it with recent changes, but was working on this just now anyway.

eburimu commented 5 years ago

Hello @globin,

I cherry pickd the PR, and when I try the following:

9996 [void@fermat:~/pi]$ cat ./libx11.sh
  9997 #!/bin/sh
  9998 nix-build -I nixpkgs=/home/void/nixpkgs -E 'let t1 = import <nixpkgs> {}; in let pkgs = import <nixpkgs> { crossSystem = t1.lib.systems.examples.armv7l-hf-multiplatform // { system = "armv7l-linux"; }; }; in pkgs.xorg.libX11 ' --show-trace

I get:

9839 [void@fermat:~/pi]$ sh ./libx11.sh
 9840 these derivations will be built:
 9841   /nix/store/x2v1gfq05dl2g5b3pj6d26pbc8xmhnyx-libX11-1.6.7-armv7l-unknown-linux-gnueabihf.drv
 9842 building '/nix/store/x2v1gfq05dl2g5b3pj6d26pbc8xmhnyx-libX11-1.6.7-armv7l-unknown-linux-gnueabihf.drv'...
 9843 unpacking sources
 9844 unpacking source archive /nix/store/rwws4hxmqrcy1rq0brbq6rvwdqa3pvsw-libX11-1.6.7.tar.bz2
 9845 source root is libX11-1.6.7
 9846 setting SOURCE_DATE_EPOCH to timestamp 1539095244 of file libX11-1.6.7/src/config.h.in
 9847 patching sources
 9848 updateAutotoolsGnuConfigScriptsPhase
 9849 Updating Autotools / GNU config script to a newer upstream version: ./config.sub
 9850 Updating Autotools / GNU config script to a newer upstream version: ./config.guess
 9851 configuring
 9852 fixing libtool script ./ltmain.sh
 9853 configure flags: --disable-static --disable-dependency-tracking --prefix=/nix/store/anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7l-unknown-linux-gnueabihf --bindir=/nix/store/anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7\
      l-unknown-linux-gnueabihf/bin --sbindir=/nix/store/anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7l-unknown-linux-gnueabihf/sbin --includedir=/nix/store/jlqym32pn110257irk6pc1kprsf751fv-libX11-1.6.7-armv7l-unknown-linux-gnueabih\
      f-dev/include --oldincludedir=/nix/store/jlqym32pn110257irk6pc1kprsf751fv-libX11-1.6.7-armv7l-unknown-linux-gnueabihf-dev/include --mandir=/nix/store/ga6d8al7wwian2c7asddcvgj97ndwxm1-libX11-1.6.7-armv7l-unknown-linux-gnueabihf-man/s\
      hare/man --infodir=/nix/store/anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7l-unknown-linux-gnueabihf/share/info --docdir=/nix/store/anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7l-unknown-linux-gnueabihf/share/doc/libX11 \
      --libdir=/nix/store/anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7l-unknown-linux-gnueabihf/lib --libexecdir=/nix/store/anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7l-unknown-linux-gnueabihf/libexec --localedir=/nix/store\
      /anz8ikivvhbxaczcbczqvhjdzpmm9x74-libX11-1.6.7-armv7l-unknown-linux-gnueabihf/share/locale --enable-malloc0returnsnull --build=x86_64-unknown-linux-gnu --host=armv7l-unknown-linux-gnueabihf
 9854 checking build system type... x86_64-unknown-linux-gnu
 9855 checking host system type... armv7l-unknown-linux-gnueabihf
 9856 checking for armv7l-unknown-linux-gnueabihf-gcc... armv7l-unknown-linux-gnueabihf-gcc
 9857 checking whether the C compiler works... yes
 9858 checking for C compiler default output file name... a.out
 9859 checking for suffix of executables...
 9860 checking whether we are cross compiling... yes
 9861 checking for suffix of object files... o
 9862 checking whether we are using the GNU C compiler... yes
 9863 checking whether armv7l-unknown-linux-gnueabihf-gcc accepts -g... yes
 9864 checking for armv7l-unknown-linux-gnueabihf-gcc option to accept ISO C89... none needed
 9865 checking whether armv7l-unknown-linux-gnueabihf-gcc understands -c and -o together... yes
 9866 checking how to run the C preprocessor... armv7l-unknown-linux-gnueabihf-gcc -E
 9867 checking for grep that handles long lines and -e... /nix/store/0151vanc3gqghnkq7kskqpj180z19xnx-gnugrep-3.1/bin/grep
 9868 checking for egrep... /nix/store/0151vanc3gqghnkq7kskqpj180z19xnx-gnugrep-3.1/bin/grep -E
 9869 checking for ANSI C header files... yes
 9870 checking for sys/types.h... yes
 9871 checking for sys/stat.h... yes
 9872 checking for stdlib.h... yes
 9873 checking for string.h... yes
 9874 checking for memory.h... yes
 9875 checking for strings.h... yes
 9876 checking for inttypes.h... yes
 9877 checking for stdint.h... yes
 9878 checking for unistd.h... yes
 9879 checking minix/config.h usability... no
 9880 checking minix/config.h presence... no
 9881 checking for minix/config.h... no
 9882 checking whether it is safe to define __EXTENSIONS__... yes
 9883 checking for a BSD-compatible install... /nix/store/9dajacm6s12cgqkc15zrrqgh26a1pd83-coreutils-8.30/bin/install -c
 9884 checking whether build environment is sane... yes
 9885 checking for armv7l-unknown-linux-gnueabihf-strip... armv7l-unknown-linux-gnueabihf-strip
 9886 checking for a thread-safe mkdir -p... /nix/store/9dajacm6s12cgqkc15zrrqgh26a1pd83-coreutils-8.30/bin/mkdir -p
 9887 checking for gawk... gawk
 9888 checking whether make sets $(MAKE)... yes
 9889 checking for style of include used by make... GNU
 9890 checking whether make supports nested variables... yes
 9891 checking dependency style of armv7l-unknown-linux-gnueabihf-gcc... none
 9892 checking how to print strings... printf
 9893 checking for a sed that does not truncate output... /nix/store/bsva73kk169ma94abpknbszxzizdgp16-gnused-4.5/bin/sed
 9894 checking for fgrep... /nix/store/0151vanc3gqghnkq7kskqpj180z19xnx-gnugrep-3.1/bin/grep -F
 9895 checking for ld used by armv7l-unknown-linux-gnueabihf-gcc... armv7l-unknown-linux-gnueabihf-ld
 9896 checking if the linker (armv7l-unknown-linux-gnueabihf-ld) is GNU ld... yes
 9897 checking for BSD- or MS-compatible name lister (nm)... armv7l-unknown-linux-gnueabihf-nm
 9898 checking the name lister (armv7l-unknown-linux-gnueabihf-nm) interface... BSD nm
 9899 checking whether ln -s works... yes
 9900 checking the maximum length of command line arguments... 32768
 9901 checking how to convert x86_64-unknown-linux-gnu file names to armv7l-unknown-linux-gnueabihf format... func_convert_file_noop
 9902 checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
 9903 checking for armv7l-unknown-linux-gnueabihf-ld option to reload object files... -r
 9904 checking for armv7l-unknown-linux-gnueabihf-objdump... armv7l-unknown-linux-gnueabihf-objdump
 9905 checking how to recognize dependent libraries... pass_all
 9906 checking for armv7l-unknown-linux-gnueabihf-dlltool... armv7l-unknown-linux-gnueabihf-dlltool
 9907 checking how to associate runtime and link libraries... printf %s\n
 9908 checking for armv7l-unknown-linux-gnueabihf-ar... armv7l-unknown-linux-gnueabihf-ar
 9909 checking for archiver @FILE support... @
 9910 checking for armv7l-unknown-linux-gnueabihf-strip... (cached) armv7l-unknown-linux-gnueabihf-strip
 9911 checking for armv7l-unknown-linux-gnueabihf-ranlib... armv7l-unknown-linux-gnueabihf-ranlib
 9912 checking command to parse armv7l-unknown-linux-gnueabihf-nm output from armv7l-unknown-linux-gnueabihf-gcc object... ok
 9913 checking for sysroot... no
 9914 checking for a working dd... /nix/store/9dajacm6s12cgqkc15zrrqgh26a1pd83-coreutils-8.30/bin/dd
 9915 checking how to truncate binary pipes... /nix/store/9dajacm6s12cgqkc15zrrqgh26a1pd83-coreutils-8.30/bin/dd bs=4096 count=1
 9916 checking for armv7l-unknown-linux-gnueabihf-mt... no
 9917 checking for mt... no
 9918 checking if : is a manifest tool... no
 9919 checking for dlfcn.h... yes
 9920 checking for objdir... .libs
 9921 checking if armv7l-unknown-linux-gnueabihf-gcc supports -fno-rtti -fno-exceptions... no
 9922 checking for armv7l-unknown-linux-gnueabihf-gcc option to produce PIC... -fPIC -DPIC
 9923 checking if armv7l-unknown-linux-gnueabihf-gcc PIC flag -fPIC -DPIC works... yes
 9924 checking if armv7l-unknown-linux-gnueabihf-gcc static flag -static works... no
 9925 checking if armv7l-unknown-linux-gnueabihf-gcc supports -c -o file.o... yes
 9926 checking if armv7l-unknown-linux-gnueabihf-gcc supports -c -o file.o... (cached) yes
 9927 checking whether the armv7l-unknown-linux-gnueabihf-gcc linker (armv7l-unknown-linux-gnueabihf-ld) supports shared libraries... yes
 9928 checking whether -lc should be explicitly linked in... no
 9929 checking dynamic linker characteristics... GNU/Linux ld.so
 9930 checking how to hardcode library paths into programs... immediate
 9931 checking whether stripping libraries is possible... yes
 9932 checking if libtool supports shared libraries... yes
 9933 checking whether to build shared libraries... yes
 9934 checking whether to build static libraries... no
 9935 checking for armv7l-unknown-linux-gnueabihf-gcc option to accept ISO C99... none needed
 9936 checking whether __clang__ is declared... no
 9937 checking whether __INTEL_COMPILER is declared... no
 9938 checking whether __SUNPRO_C is declared... no
 9939 checking for armv7l-unknown-linux-gnueabihf-pkg-config... no
 9940 checking for pkg-config... /nix/store/xvn8sh7cqvmij6ah9mxsxhzq8xlxnn9l-pkg-config-0.29.2/bin/pkg-config
 9941 configure: WARNING: using cross tools not prefixed with host triplet
 9942 checking pkg-config is at least version 0.9.0... yes
 9943 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=unknown-warning-option... no
 9944 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=unused-command-line-argument... no
 9945 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wall... yes
 9946 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wpointer-arith... yes
 9947 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wmissing-declarations... yes
 9948 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wformat=2... yes
 9949 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wstrict-prototypes... yes
 9950 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wmissing-prototypes... yes
 9951 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wnested-externs... yes
 9952 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wbad-function-cast... yes
 9953 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wold-style-definition... yes
 9954 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wdeclaration-after-statement... yes
 9955 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wunused... yes
 9956 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wuninitialized... yes
 9957 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wshadow... yes
 9958 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wmissing-noreturn... yes
 9959 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wmissing-format-attribute... yes
 9960 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wredundant-decls... yes
 9961 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Wlogical-op... yes
 9962 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=implicit... yes
 9963 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=nonnull... yes
 9964 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=init-self... yes
 9965 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=main... yes
 9966 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=missing-braces... yes
 9967 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=sequence-point... yes
 9968 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=return-type... yes
 9969 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=trigraphs... yes
 9970 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=array-bounds... yes
 9971 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=write-strings... yes
 9972 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=address... yes
  9973 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=int-to-pointer-cast... yes
  9974 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=pointer-to-int-cast... yes
  9975 checking if armv7l-unknown-linux-gnueabihf-gcc supports -pedantic... yes
  9976 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror... yes
  9977 checking if armv7l-unknown-linux-gnueabihf-gcc supports -Werror=attributes... yes
  9978 Package xorg-macros was not found in the pkg-config search path.
  9979 Perhaps you should add the directory containing `xorg-macros.pc'
  9980 to the PKG_CONFIG_PATH environment variable
  9981 No package 'xorg-macros' found
  9982 checking whether make supports nested variables... (cached) yes
  9983 checking whether to build functional specifications... yes
  9984 checking for xmlto... no
  9985 configure: WARNING: xmlto not found - documentation targets will be skipped
  9986 checking for xsltproc... no
  9987 configure: WARNING: xsltproc not found - cannot transform XML documents
  9988 checking for X.Org SGML entities >= 1.10... no
  9989 checking for cpp... no
  9990 checking if armv7l-unknown-linux-gnueabihf-gcc -E requires -undef... armv7l-unknown-linux-gnueabihf-gcc: fatal error: no input files
  9991 compilation terminated.
  9992 armv7l-unknown-linux-gnueabihf-gcc: fatal error: no input files
  9993 compilation terminated.
  9994 armv7l-unknown-linux-gnueabihf-gcc: fatal error: no input files
  9995 compilation terminated.
  9996 configure: error: armv7l-unknown-linux-gnueabihf-gcc -E defines unix with or without -undef.  I don't know what to do.
  9997 builder for '/nix/store/x2v1gfq05dl2g5b3pj6d26pbc8xmhnyx-libX11-1.6.7-armv7l-unknown-linux-gnueabihf.drv' failed with exit code 1
  9998 error: build of '/nix/store/x2v1gfq05dl2g5b3pj6d26pbc8xmhnyx-libX11-1.6.7-armv7l-unknown-linux-gnueabihf.drv' failed

Any ideas? Maybe I'm missing some patches.

globin commented 5 years ago

I tried with aarch64, maybe armv7 needs some more fixes..

eburimu commented 5 years ago

From the output, I see that it requires pkgconfig and libxslt as nativeBuildInputs. I can try if you add those.

globin commented 5 years ago

Actually it worked for me with:

nix-build --arg crossSystem '(import ./. {}).lib.systems.examples.armv7l-hf-multiplatform' -A xorg.libX11

But the error is the same mentioned in https://github.com/NixOS/nixpkgs/pull/51180#discussion_r237349993, which is a deeper issue.

eburimu commented 5 years ago

Hey @globin,

depsBuildBuils = [ buildPackages.stdenv.cc ];

seems to fix the cpp problem. Not sure if its gonna work when I move it to the target, thnx anyway.

eburimu commented 5 years ago

See https://github.com/NixOS/nixpkgs/pull/51180#discussion_r237349993