search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.99k stars 14.01k forks source link

prusa-slicer: build failure since staging-next 2022-07-29 #185808

Closed gador closed 2 years ago

gador commented 2 years ago

Steps To Reproduce

Steps to reproduce the behavior:

  1. build prusa-slicer with current master

Build log

build log on hydra

Additional context

I did a git bisect --first-parent and found #183748 (with commit a0ebaa74318ec9a9e3cdb561c1b4ecce79e0054c) to be the first bad commit. Looking through the commits on that staging-next run, I tried reverting e682dd84cff5d2420fcc0a40508557477f6cc9d3 and could build prusa-slicer again.

So it seems to be a regression caused in binutils-wrapper

Notify maintainers

@Artturin (author of commit) @vcunat (staging-next run)

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.56, NixOS, 22.11 (Raccoon), 22.11.20220729.99d2206`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.10.3`
 - channels(root): `"nixos, nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
vcunat commented 2 years ago

Package maintainers: @Moredread, @thorstenweber83. They should at least know it's failing.

gador commented 2 years ago

Ah, thanks, sorry I forgot

Artturin commented 2 years ago 
< cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_LOCALEDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/share/locale -DCMAKE_INSTALL_LIB
EXECDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/share/doc/PrusaSlicer -DCMAKE
_INSTALL_INFODIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/incl
ude -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/include -DCMAKE_INSTALL_SBINDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/bin
-DCMAKE_INSTALL_NAME_DIR=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/2vqbkw1s50pvk970k06wkzialgk7jp7v-gcc-wrapper-11.3.0/bin/strip -DCMAKE_RANLIB=/nix/s
tore/ydvvnn6mi99kyp9an1gnrcga1dr5v5x0-binutils-2.38/bin/ranlib -DCMAKE_AR=/nix/store/ydvvnn6mi99kyp9an1gnrcga1dr5v5x0-binutils-2.38/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/jg81vm6ia0mpn09hrnd72ps56yl4c3vx-prusa-slicer-2.4.2 -DSLIC3R_FH
S=1 -DSLIC3R_GTK=3
---
> cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_LOCALEDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/share/locale -DCMAKE_INSTALL_LIB
EXECDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/share/doc/PrusaSlicer -DCMAKE
_INSTALL_INFODIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/incl
ude -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/include -DCMAKE_INSTALL_SBINDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/bin
-DCMAKE_INSTALL_NAME_DIR=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/strip -DCMAKE_RANLIB=/nix/s
tore/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/ranlib -DCMAKE_AR=/nix/store/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/494h5s7wknjcs4mzg8l68xx7di2dwcfy-prusa-slicer-2.4.2 -
DSLIC3R_FHS=1 -DSLIC3R_GTK=3

Previous working https://hydra.nixos.org/build/185878785

-DCMAKE_STRIP=/nix/store/2vqbkw1s50pvk970k06wkzialgk7jp7v-gcc-wrapper-11.3.0/bin/strip -DCMAKE_RANLIB=/nix/s
tore/ydvvnn6mi99kyp9an1gnrcga1dr5v5x0-binutils-2.38/bin/ranlib -DCMAKE_AR=/nix/store/ydvvnn6mi99kyp9an1gnrcga1dr5v5x0-binutils-2.38/bin/ar

Broken https://hydra.nixos.org/build/186156640

-DCMAKE_STRIP=/nix/store/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/strip -DCMAKE_RANLIB=/nix/s
tore/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/ranlib -DCMAKE_AR=/nix/store/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/ar

https://github.com/NixOS/nixpkgs/blob/bbd9e0bfd03cfedaa17f9360daa2f39e613b40b2/pkgs/development/tools/build-managers/cmake/setup-hook.sh#L45-L47

i don't see other differences related to binutils, weird.

Artturin commented 2 years ago

prusa-slicer is also quite heavy to build, it would be easier to debug if there was a lighter package i could debug with

vojta001 commented 2 years ago

Just to add my two cents:

It is not failing for me, it is just not going anywhere. After more than 12 hours, there is still the following process eating a single core of my CPU.

/nix/store/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/as -I /nix/store/wxsyxbr2nr2dccw5gsc5xabkjv00g9q7-dbus-1.14.0-dev/include/dbus-1.0 -I /nix/store/0zs6d6hkmb8z32z2q8f60qv0hx39ql66-dbus-1.14.0-lib/lib/dbus-1.0/include -I /build/source/src -I /build/source/build/src/platform -I /build/source/tests -I /build/source/build/src/libslic3r -I /build/source/src/libnest2d/include -I /build/source/src/miniz -I /build/source/src/glu-libtess/include -I /build/source/src/qhull/src --gdwarf-5 --64 --compress-debug-sections -o CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.o /build/cchQACR0.s
gador commented 2 years ago

I face the same issue. It just hogs 100% of a single core with the assembler without going anywhere

vcunat commented 2 years ago

Well yes, same on Hydra – it always says "Timed out", not normal failure.

Artturin commented 2 years ago

nixos-22.05

/nix/store/ag2bpk0lzjvj409znklrz5krkpc5imzs-gcc-wrapper-11.3.0/bin/as@ ⇒ /nix/stor/wp7kc5y840ldi5sd8j81fj4dabccff2d-binutils-wrapper-2.38/bin/as

master

/nix/store/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/as@ ⇒ /nix/stor/klgmnpyyan0gg6ra6d40qmjqs39cj1cf-binutils-wrapper-2.38/bin/as

so no difference there

the only difference is that gcc-wrapper contains more symlinks to binutils because the wrapper now has all of them https://github.com/NixOS/nixpkgs/blob/33c6cdd2d1e89d657a844556ebc6c7df02b87791/pkgs/build-support/cc-wrapper/default.nix#L197-L203

Moredread commented 2 years ago

I'm using ccache for debugging compilation problems, esp. for prusa-slicer. If it hangs at the same point everytime, this might save you a lot of time on recompiles. https://nixos.wiki/wiki/CCache

Moredread commented 2 years ago

Just to add my two cents:

It is not failing for me, it is just not going anywhere. After more than 12 hours, there is still the following process eating a single core of my CPU.

/nix/store/kia8xygy2r9iblwbjrl163j4bpz84wz7-gcc-wrapper-11.3.0/bin/as -I /nix/store/wxsyxbr2nr2dccw5gsc5xabkjv00g9q7-dbus-1.14.0-dev/include/dbus-1.0 -I /nix/store/0zs6d6hkmb8z32z2q8f60qv0hx39ql66-dbus-1.14.0-lib/lib/dbus-1.0/include -I /build/source/src -I /build/source/build/src/platform -I /build/source/tests -I /build/source/build/src/libslic3r -I /build/source/src/libnest2d/include -I /build/source/src/miniz -I /build/source/src/glu-libtess/include -I /build/source/src/qhull/src --gdwarf-5 --64 --compress-debug-sections -o CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.o /build/cchQACR0.s

Is it hanging at the same point each time? In this case it is "just" a test that fails, so on one hand limiting compilation to just the test suite could help narrowing it down (the tests are compiled during buildPhase atm as I haven't figured out how cmake targets interact with the build and test phases during build).

On the other hand disabling the test module in the cmake config might give an indication of the compilation works otherwise.

Edit:

I've taken a look and it also happens on my machine.

To just build the problematic file add the following to the prusa-slicer derivation

  preBuild = ''
    cd tests/libslic3r
    time make test_hollowing.cpp.o
    time make test_voronoi.cpp.o
    exit 1
  '';

It should just build two of the test files, one that should build just fine and the one that hangs. It also times the build. For rev 70b75dd824ae0bf16f06f88783dbb117f8047bcd (a previously working rev) building takes about 55s on my machine.

prusa-slicer> Building CXX object tests/libslic3r/CMakeFiles/libslic3r_tests.dir/test_hollowing.cpp.o
prusa-slicer> real      0m4.277s
prusa-slicer> user      0m3.969s
prusa-slicer> sys       0m0.292s
prusa-slicer> Building CXX object tests/libslic3r/CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.o
prusa-slicer> real      0m54.836s
prusa-slicer> user      0m54.125s
prusa-slicer> sys       0m0.689s
Moredread commented 2 years ago

I've added the build tools for the reverted commit (https://github.com/NixOS/nixpkgs/commit/a0ebaa74318ec9a9e3cdb561c1b4ecce79e0054c and https://github.com/NixOS/nixpkgs/commit/e682dd84cff5d2420fcc0a40508557477f6cc9d3 reverted) to my cachix cache, as it needs a complete rebuild otherwise. See https://github.com/Moredread/nixpkgs/tree/prusa-slicer-staging-revert for the matching nixpkgs tree.

https://app.cachix.org/cache/moredread#pull describes how to use the cache.

Moredread commented 2 years ago

I'll describe my further debugging efforts here and might edit the post if I find something new:

Running

    cd tests/libslic3r
    make test_voronoi.cpp.i
    make test_voronoi.cpp.s
    wc CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.i
    wc CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.s

shows for bad (the https://github.com/NixOS/nixpkgs/commit/a0ebaa74318ec9a9e3cdb561c1b4ecce79e0054c commit) and good (https://github.com/NixOS/nixpkgs/commit/e682dd84cff5d2420fcc0a40508557477f6cc9d3 reverted)

good:

prusa-slicer>  219085  590541 6977122 CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.i
prusa-slicer>   861315  1666255 13943712 CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.s

bad:

prusa-slicer>  219085  590541 6977122 CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.i
prusa-slicer>  20531344  41439102 365065283 CMakeFiles/libslic3r_tests.dir/test_voronoi.cpp.s

I.e. the assembly file is a lot larger for bad.

gador commented 2 years ago

Some things I have tested:

When adding -DSLIC3R_ASAN=ON it adds the Google AddressSanitizer. This tool acts as an memory error detector (like leaks etc). When also adding ASAN_OPTIONS=detect_leaks=false as suggested here prusa-slicer actually builds! Unfortunately, I get a lot of error messages at startup and lots of ASan Debug messages in the console on exit. So not really a working solution. Why adding this helps in building? I have no clue. It only adds the necessary compile flags for ASan: https://github.com/prusa3d/PrusaSlicer/blob/991f2bdbd18cbc18912147d7886bb1e1e3412742/CMakeLists.txt#L258-L273

When adding sed -i 's|test_voronoi.cpp||g' tests/libslic3r/CMakeLists.txt to prePatch to exclude the test, prusa-slicer builds and seems to be working normally.

vojta001 commented 2 years ago

I tried building it in staging-next (it contains #182385, which @mweinelt referenced), but it got stuck as well

Moredread commented 2 years ago

It seems to me that there is a subtle difference with the binutils-wrapper change to how the compiler outputs assembly. I'm not sure though how gcc handles the invocation of the build pipeline internally (I think I remember that it changed a few years ago) when it is asked to output an object file vs i.e. assembly and if it should make a difference if you invoke them manually.

When adding sed -i 's|test_voronoi.cpp||g' tests/libslic3r/CMakeLists.txt to prePatch to exclude the test, prusa-slicer builds and seems to be working normally.

I'll create a PR with this change for now, so we get the derivation working for now, but it seems there is a deeper issue here.

baduhai commented 2 years ago

It is now building again as per hydra status. I think this can be closed.

gador commented 2 years ago

I agree. The underlying cause was never found, but disabling the test worked and I haven't experienced any regressions