search

JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.73k stars 5.48k forks source link

Unable to build Julia 1.9.0-beta2 from source #48235

Closed kpamnany closed 1 year ago

kpamnany commented 1 year ago

The Julia 1.9.0-beta2 source tarballs (both without and with dependencies) are broken for us (see log below). It appears that the bundled lld fails as the interpreter of the downloaded binary points to /lib64/ld-linux-x86-64.so.2 which isn't present in our build environment on Nix. I've tried both archives, with and without USE_BINARYBUILDER=0.

We've temporarily fixed this by patching base/linking.jl to remove this line, thereby forcing use of the system lld.

    JULIA usr/lib/julia/sys-o.a
ERROR: IOError: could not spawn setenv(`/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/tools/lld -flavor gnu '' -shared -o /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/jl_sdFMzW/compiled/v1.9/jl_Oc0G1c --whole-archive /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/jl_sdFMzW/compiled/v1.9/jl_UVOvEh --no-whole-archive -L/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib -L/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib/julia -L/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib -ljulia -ljulia-internal`,["MFLAGS=-s", "CXX=g++ -m64", "postInstall=patchelf --set-interpreter \$LI --set-rpath \$out/lib/julia \$out/libexec/7z\npatchelf --set-interpreter \$LI --set-rpath \$out/lib:\$out/lib/julia \$out/bin/julia\npatchelf --set-rpath \$out/lib/julia \$out/lib/julia/libgfortran.so.*.*.*\n", "HOST_PATH=/nix/store/abbckp4y0lqr8qds4h301j66w5mi3bn5-gnum4-1.4.19/bin:/nix/store/4b0ybsv7cafx94wlc1vwjmz4cjkqkfpc-automake-1.16.3/bin:/nix/store/5b45mbx715pqsvcli8w1w6s383hrhvmi-autoconf-2.71/bin:/nix/store/2pfr3v0zwsp2nzw7v9qqvagi430ajrn8-pkg-config-wrapper-0.29.2/bin:/nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/bin:/nix/store/jkqrnwq5wppfxfpg8nnav7mh4y0xy1ij-gfortran-wrapper-9.3.0/bin:/nix/store/jc94ssw4g1dj1r4130z9irjpb413j0mf-binutils-wrapper-2.35.2/bin:/nix/store/08zrhnlprkp2wk294pfmf3vy0fxnrrny-which-2.21/bin:/nix/store/550j7ajkrm7j548f98fya65xi2xl19vf-perl-5.34.0/bin:/nix/store/why2v7sy6cgpc7m4qlj8qxppmh0n5512-openssl-1.1.1o-bin/bin:/nix/store/aqgpfay84b43lkrknbvxqqisv9wg633x-curl-7.79.1-dev/bin:/nix/store/dp8sjb0bdymg814vw36vh4mh01d8qs05-nghttp2-1.43.0-bin/bin:/nix/store/3qzaahipydyj5k6bmw0zj2a3yml891wf-libidn-1.38-bin/bin:/nix/store/1dd7d04396rs759smvp4lpw70ci6lbsg-libkrb5-1.18-dev/bin:/nix/store/sxwqxjcqjxqa25chy8k4lg6di2lsy2zk-libkrb5-1.18/bin:/nix/store/qdy074pisnmq1k6mdp6wfr2x1kdy63ai-brotli-1.0.9/bin:/nix/store/m7h45gwlik1nyd4acf2p07qcsz7ivawm-curl-7.79.1-bin/bin:/nix/store/i9q0jv6qnvg7zal98rqi7aq31k3p89hw-coreutils-9.0/bin:/nix/store/dykv06vzcn128bmwi19fm640xkdp747f-findutils-4.8.0/bin:/nix/store/58yya3k69fkk5x0gfazw1h3qjf1plgii-diffutils-3.8/bin:/nix/store/mbj7jqg5cgzsc8nml56n3x4ymq3gca5q-gnused-4.8/bin:/nix/store/kil8r0780pvzzm2kfljvwd407fdcfa2k-gnugrep-3.7/bin:/nix/store/cznzsql8cs7nv6g6gflf1724r7d4s9lz-gawk-5.1.1/bin:/nix/store/gapgxqfyny37b2i6xbcmdzb9vnqar6y0-gnutar-1.34/bin:/nix/store/296s5ywm9hw8ia9hr9j80z47rv51v6y9-gzip-1.12/bin:/nix/store/kwk9gwaly79cwkmv4msd8rj8hyva299m-bzip2-1.0.6.0.2-bin/bin:/nix/store/p30hamqm7fn0i7p8fq81d0ijv0gg0b9g-gnumake-4.3/bin:/nix/store/dfqckm5is1k13q24sv1hxpmj2yhlsjlj-bash-5.1-p8/bin:/nix/store/k51ba3zyi52q66yyvc4lba96ndc3fvx0-patch-2.7.6/bin:/nix/store/9wvdg6ayndlmhan3ccz938l0k9hvgxg9-xz-5.2.5-bin/bin", "doInstallCheck=", "PATH=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/tools:/nix/store/fdjqirgff4vyv92vqa2b59fr1zppj45n-patchelf-0.13/bin:/nix/store/myjrqkb583ak29f3r5wmxrazy33i90pi-gcc-wrapper-10.3.0/bin:/nix/store/gwdq0r345gjpnsbllqjgrb8c8z23aiw9-gcc-10.3.0/bin:/nix/store/al0y3y2668icw7mzhagbb8ywg93dgnzh-glibc-2.33-123-bin/bin:/nix/store/i9q0jv6qnvg7zal98rqi7aq31k3p89hw-coreutils-9.0/bin:/nix/store/8rmwwv30162vnhn5l74m0vjiyq7dnm90-binutils-wrapper-2.35.2/bin:/nix/store/8pjwvcn5513hgq7g0v1192w17jsgn7k8-binutils-2.35.2/bin:/nix/store/abbckp4y0lqr8qds4h301j66w5mi3bn5-gnum4-1.4.19/bin:/nix/store/4b0ybsv7cafx94wlc1vwjmz4cjkqkfpc-automake-1.16.3/bin:/nix/store/5b45mbx715pqsvcli8w1w6s383hrhvmi-autoconf-2.71/bin:/nix/store/2pfr3v0zwsp2nzw7v9qqvagi430ajrn8-pkg-config-wrapper-0.29.2/bin:/nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/bin:/nix/store/jkqrnwq5wppfxfpg8nnav7mh4y0xy1ij-gfortran-wrapper-9.3.0/bin:/nix/store/jc94ssw4g1dj1r4130z9irjpb413j0mf-binutils-wrapper-2.35.2/bin:/nix/store/08zrhnlprkp2wk294pfmf3vy0fxnrrny-which-2.21/bin:/nix/store/550j7ajkrm7j548f98fya65xi2xl19vf-perl-5.34.0/bin:/nix/store/why2v7sy6cgpc7m4qlj8qxppmh0n5512-openssl-1.1.1o-bin/bin:/nix/store/aqgpfay84b43lkrknbvxqqisv9wg633x-curl-7.79.1-dev/bin:/nix/store/dp8sjb0bdymg814vw36vh4mh01d8qs05-nghttp2-1.43.0-bin/bin:/nix/store/3qzaahipydyj5k6bmw0zj2a3yml891wf-libidn-1.38-bin/bin:/nix/store/1dd7d04396rs759smvp4lpw70ci6lbsg-libkrb5-1.18-dev/bin:/nix/store/sxwqxjcqjxqa25chy8k4lg6di2lsy2zk-libkrb5-1.18/bin:/nix/store/qdy074pisnmq1k6mdp6wfr2x1kdy63ai-brotli-1.0.9/bin:/nix/store/m7h45gwlik1nyd4acf2p07qcsz7ivawm-curl-7.79.1-bin/bin:/nix/store/i9q0jv6qnvg7zal98rqi7aq31k3p89hw-coreutils-9.0/bin:/nix/store/dykv06vzcn128bmwi19fm640xkdp747f-findutils-4.8.0/bin:/nix/store/58yya3k69fkk5x0gfazw1h3qjf1plgii-diffutils-3.8/bin:/nix/store/mbj7jqg5cgzsc8nml56n3x4ymq3gca5q-gnused-4.8/bin:/nix/store/kil8r0780pvzzm2kfljvwd407fdcfa2k-gnugrep-3.7/bin:/nix/store/cznzsql8cs7nv6g6gflf1724r7d4s9lz-gawk-5.1.1/bin:/nix/store/gapgxqfyny37b2i6xbcmdzb9vnqar6y0-gnutar-1.34/bin:/nix/store/296s5ywm9hw8ia9hr9j80z47rv51v6y9-gzip-1.12/bin:/nix/store/kwk9gwaly79cwkmv4msd8rj8hyva299m-bzip2-1.0.6.0.2-bin/bin:/nix/store/p30hamqm7fn0i7p8fq81d0ijv0gg0b9g-gnumake-4.3/bin:/nix/store/dfqckm5is1k13q24sv1hxpmj2yhlsjlj-bash-5.1-p8/bin:/nix/store/k51ba3zyi52q66yyvc4lba96ndc3fvx0-patch-2.7.6/bin:/nix/store/9wvdg6ayndlmhan3ccz938l0k9hvgxg9-xz-5.2.5-bin/bin:/nix/store/c19pfwa36mw6xb30f2vi105nvs5956lv-cmake-3.21.2/bin", "BUILDROOT=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source", "MAKEFLAGS=s -- SHELL=/nix/store/dfqckm5is1k13q24sv1hxpmj2yhlsjlj-bash-5.1-p8/bin/bash prefix=\$\$(out) USE_BINARYBUILDER=1 USE_GPL_LIBS=0 ARCH=x86_64 JULIA_CPU_TARGET=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1) LLVM_LDFLAGS=-lLLVM", "MAKEOVERRIDES=\${-*-command-variables-*-}", "CC=gcc -m64", "NIX_BINTOOLS_WRAPPER_TARGET_TARGET_x86_64_unknown_linux_gnu=1", "ARCH=x86_64", "_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__linux_x86_64-linux-gnu", "outputs=out", "LD_LIBRARY_PATH=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib/julia:/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib::/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib", "OBJDUMP_FOR_TARGET=objdump", "PERL5LIB=/nix/store/550j7ajkrm7j548f98fya65xi2xl19vf-perl-5.34.0/lib/perl5/site_perl", "dontStrip=1", "TMP=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0", "OBJCOPY=objcopy", "READELF_FOR_TARGET=readelf", "SSL_CERT_FILE=/nix/store/zvg95qpq4axhm9qp3da3gqjgjijhbxzb-nss-cacert-3.77/etc/ssl/certs/ca-bundle.crt", "makeFlags=prefix=\$(out) USE_BINARYBUILDER=1 USE_GPL_LIBS=0 ARCH=x86_64 JULIA_CPU_TARGET=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1) LLVM_LDFLAGS=-lLLVM", "lt_cv_deplibs_check_method=pass_all", "NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu=1", "NIX_CC_WRAPPER_TARGET_TARGET_x86_64_unknown_linux_gnu=1", "SYSTEM_CERTIFICATE_PATH=/nix/store/zvg95qpq4axhm9qp3da3gqjgjijhbxzb-nss-cacert-3.77/etc/ssl/certs/ca-bundle.crt", "LD_FOR_TARGET=ld", "depsBuildTargetPropagated=", "LD=ld", "PYTHONHASHSEED=0", "NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu=1", "enableParallelBuilding=", "STRIP_FOR_TARGET=strip", "NM=nm", "MAKELEVEL=2", "STRIP=strip", "NIX_LDFLAGS_FOR_TARGET= -L/nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/lib -L/nix/store/2yy85x1bhwmynzmpr4n29caxpfm0bkk4-zlib-1.2.12/lib -L/nix/store/fmxkxd21yf4qklf4b07p8va881m0giq8-nghttp2-1.43.0-lib/lib -L/nix/store/ffa40clh365ccc4r47d4rkw5sz8i00gk-libidn-1.38/lib -L/nix/store/sxwqxjcqjxqa25chy8k4lg6di2lsy2zk-libkrb5-1.18/lib -L/nix/store/dmhq727zgz5zixq1vz22kqji5y1bgm5i-openssl-1.1.1o/lib -L/nix/store/drbyp7dgw4bi760zsyy6b7hzxp32zj28-libssh2-1.10.0/lib -L/nix/store/3gjnn0qidg12aqb9byjnp7f563mkv3s3-brotli-1.0.9-lib/lib -L/nix/store/1blwp2kqjg5xkk2gs3i3l8l36r1mhzqi-curl-7.79.1/lib", "XDG_DATA_DIRS=/nix/store/fdjqirgff4vyv92vqa2b59fr1zppj45n-patchelf-0.13/share", "NIX_LOG_FD=2", "OBJDUMP=objdump", "nativeBuildInputs=", "SHELL=/nix/store/dfqckm5is1k13q24sv1hxpmj2yhlsjlj-bash-5.1-p8/bin/bash", "PKG_CONFIG_PATH_FOR_TARGET=/nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/lib/pkgconfig:/nix/store/2y10rqa796j9xcjai8sqi9xa3gvaifh8-zlib-1.2.12-dev/lib/pkgconfig:/nix/store/aqgpfay84b43lkrknbvxqqisv9wg633x-curl-7.79.1-dev/lib/pkgconfig:/nix/store/1gi5whc298kbxqjv57j31lz5div9cpc6-nghttp2-1.43.0-dev/lib/pkgconfig:/nix/store/b2jgp8hm35k9420ba00dm8xdnvkpd31d-libidn-1.38-dev/lib/pkgconfig:/nix/store/1dd7d04396rs759smvp4lpw70ci6lbsg-libkrb5-1.18-dev/lib/pkgconfig:/nix/store/3hpiypawj4wwcbwi37wp8clky96dcc8y-openssl-1.1.1o-dev/lib/pkgconfig:/nix/store/xhbg31f6vfv1hcpndqv18j9igqv1fsms-libssh2-1.10.0-dev/lib/pkgconfig:/nix/store/881nfkxjjf1dymbr020yk0wmsbaf4baz-brotli-1.0.9-dev/lib/pkgconfig", "system=x86_64-linux", "depsBuildBuild=", "name=julia-1.9.0-beta2-patched", "requiredSystemFeatures=big-parallel", "AS_FOR_TARGET=as", "RANLIB=ranlib", "TMPDIR=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0", "USE_GPL_LIBS=0", "NIX_SSL_CERT_FILE=/nix/store/zvg95qpq4axhm9qp3da3gqjgjijhbxzb-nss-cacert-3.77/etc/ssl/certs/ca-bundle.crt", "buildInputs=/nix/store/abbckp4y0lqr8qds4h301j66w5mi3bn5-gnum4-1.4.19 /nix/store/4b0ybsv7cafx94wlc1vwjmz4cjkqkfpc-automake-1.16.3 /nix/store/5b45mbx715pqsvcli8w1w6s383hrhvmi-autoconf-2.71 /nix/store/2pfr3v0zwsp2nzw7v9qqvagi430ajrn8-pkg-config-wrapper-0.29.2 /nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13 /nix/store/jkqrnwq5wppfxfpg8nnav7mh4y0xy1ij-gfortran-wrapper-9.3.0 /nix/store/08zrhnlprkp2wk294pfmf3vy0fxnrrny-which-2.21 /nix/store/550j7ajkrm7j548f98fya65xi2xl19vf-perl-5.34.0 /nix/store/why2v7sy6cgpc7m4qlj8qxppmh0n5512-openssl-1.1.1o-bin /nix/store/zvg95qpq4axhm9qp3da3gqjgjijhbxzb-nss-cacert-3.77 /nix/store/2y10rqa796j9xcjai8sqi9xa3gvaifh8-zlib-1.2.12-dev /nix/store/aqgpfay84b43lkrknbvxqqisv9wg633x-curl-7.79.1-dev", "NIX_BUILD_TOP=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0", "depsHostHostPropagated=", "depsTargetTargetPropagated=", "doCheck=", "depsBuildTarget=", "SOURCE_DATE_EPOCH=315532800", "NIX_PKG_CONFIG_WRAPPER_TARGET_TARGET_x86_64_unknown_linux_gnu=1", "STRINGS=strings", "HOME=/homeless-shelter", "TERM=xterm-256color", "__noChroot=1", "AR_FOR_TARGET=ar", "TEMP=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0", "PYTHONNOUSERSITE=1", "_PYTHON_HOST_PLATFORM=linux-x86_64", "NIX_CFLAGS_COMPILE=-L/nix/store/6v1dixpdy65bqmafq642nbv9490a9c10-gfortran-9.3.0-lib/lib -frandom-seed=1549kfhbrs -isystem /nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/include -isystem /nix/store/2y10rqa796j9xcjai8sqi9xa3gvaifh8-zlib-1.2.12-dev/include -isystem /nix/store/aqgpfay84b43lkrknbvxqqisv9wg633x-curl-7.79.1-dev/include -isystem /nix/store/1gi5whc298kbxqjv57j31lz5div9cpc6-nghttp2-1.43.0-dev/include -isystem /nix/store/b2jgp8hm35k9420ba00dm8xdnvkpd31d-libidn-1.38-dev/include -isystem /nix/store/1dd7d04396rs759smvp4lpw70ci6lbsg-libkrb5-1.18-dev/include -isystem /nix/store/3hpiypawj4wwcbwi37wp8clky96dcc8y-openssl-1.1.1o-dev/include -isystem /nix/store/xhbg31f6vfv1hcpndqv18j9igqv1fsms-libssh2-1.10.0-dev/include -isystem /nix/store/881nfkxjjf1dymbr020yk0wmsbaf4baz-brotli-1.0.9-dev/include -isystem /nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/include -isystem /nix/store/2y10rqa796j9xcjai8sqi9xa3gvaifh8-zlib-1.2.12-dev/include -isystem /nix/store/aqgpfay84b43lkrknbvxqqisv9wg633x-curl-7.79.1-dev/include -isystem /nix/store/1gi5whc298kbxqjv57j31lz5div9cpc6-nghttp2-1.43.0-dev/include -isystem /nix/store/b2jgp8hm35k9420ba00dm8xdnvkpd31d-libidn-1.38-dev/include -isystem /nix/store/1dd7d04396rs759smvp4lpw70ci6lbsg-libkrb5-1.18-dev/include -isystem /nix/store/3hpiypawj4wwcbwi37wp8clky96dcc8y-openssl-1.1.1o-dev/include -isystem /nix/store/xhbg31f6vfv1hcpndqv18j9igqv1fsms-libssh2-1.10.0-dev/include -isystem /nix/store/881nfkxjjf1dymbr020yk0wmsbaf4baz-brotli-1.0.9-dev/include", "DETERMINISTIC_BUILD=1", "patches=/nix/store/q10i5cr9h3bbg08zfsl405kfhhdxf2kk-version-rai.patch /nix/store/bfw1pn4g5m3hqz21h24553lck2j1ay4q-jl-pool-alloc.patch /nix/store/g4b7hv3jpmqlgpzxras0r3bg1c29nbzp-47615-manual.patch /nix/store/adg0xgwpf07ym2hl0mb011s842jgdzqz-task-priority.patch /nix/store/b2b7zw733k6ar0fmvbdbvh7k6yswxip4-total-ttsp.patch", "READELF=readelf", "AS=as", "RANLIB_FOR_TARGET=ranlib", "depsHostHost=", "TEMPDIR=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0", "NIX_INDENT_MAKE=1", "strictDeps=", "shell=/nix/store/dfqckm5is1k13q24sv1hxpmj2yhlsjlj-bash-5.1-p8/bin/bash", "LC_ALL=C", "TZ=UTC", "NIX_BINTOOLS=/nix/store/8rmwwv30162vnhn5l74m0vjiyq7dnm90-binutils-wrapper-2.35.2", "FC_FOR_TARGET=gfortran", "src=/nix/store/rd8g5kqkqycp8w4k9zl5l91j5icbil42-source", "JULIA_BINDIR=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/bin", "STRINGS_FOR_TARGET=strings", "SIZE=size", "prefix=/nix/store/1549kfhbrsiad70d7mjvix9a80frxr0s-julia-1.9.0-beta2-patched", "NIX_BINTOOLS_FOR_TARGET=/nix/store/jc94ssw4g1dj1r4130z9irjpb413j0mf-binutils-wrapper-2.35.2", "version=1.9.0-beta2", "LANG=C", "LI=/nix/store/0xxjx37fcy2nl3yz6igmv4mag2a7giq6-glibc-2.33-123/lib/ld-linux-x86-64.so.2", "ACLOCAL_PATH=/nix/store/4b0ybsv7cafx94wlc1vwjmz4cjkqkfpc-automake-1.16.3/share/aclocal:/nix/store/2pfr3v0zwsp2nzw7v9qqvagi430ajrn8-pkg-config-wrapper-0.29.2/share/aclocal:/nix/store/aqgpfay84b43lkrknbvxqqisv9wg633x-curl-7.79.1-dev/share/aclocal", "SHLVL=2", "OLDPWD=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source", "NIX_CC=/nix/store/myjrqkb583ak29f3r5wmxrazy33i90pi-gcc-wrapper-10.3.0", "_=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/bin/julia", "MAKE_TERMERR=/dev/pts/0", "configureFlags=--prefix=/nix/store/1549kfhbrsiad70d7mjvix9a80frxr0s-julia-1.9.0-beta2-patched ", "depsTargetTarget=", "AR=ar", "SIZE_FOR_TARGET=size", "JULIA_NUM_THREADS=1", "PKG_CONFIG_FOR_TARGET=pkg-config", "noAuditTmpdir=1", "NIX_ENFORCE_NO_NATIVE=1", "preConfigure=export PATH=\$PATH:/nix/store/c19pfwa36mw6xb30f2vi105nvs5956lv-cmake-3.21.2/bin\nsed -e '/^install:/s@[^ ]*/doc/[^ ]*@@' -i Makefile\nsed -e '/[\$](DESTDIR)[\$](docdir)/d' -i Makefile\n# downloaded llvm-config either needs patchelf patching, or like we do here, called using\n# an absolute path to the interpreter (on normal linux: /lib64/ld-linux.so.2)\nexport LI=\"\$(patchelf --print-interpreter \$(type -P bash))\"\nsed -i 's|\$(LLVM_CONFIG_HOST)|'\$LI' \$(LLVM_CONFIG_HOST)|' Make.inc src/Makefile base/Makefile\n\n# RV: Super ugly workaround for a RPATH issue where downloaded\n# libgfortran cant find libquadmath in same dir. The libgfortran\n# lib is later in this build patched to have proper rpath.\nexport LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:\$PWD/usr/lib\n", "PWD=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/base", "NIX_STORE=/nix/store", "NIX_BUILD_CORES=32", "PKG_CONFIG_LIBDIR=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib/pkgconfig", "PYTHONPATH=/nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/lib/python3.9/site-packages", "OBJCOPY_FOR_TARGET=objcopy", "OPENBLAS_NUM_THREADS=1", "out=/nix/store/1549kfhbrsiad70d7mjvix9a80frxr0s-julia-1.9.0-beta2-patched", "WINEPATH=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/bin;", "builder=/nix/store/dfqckm5is1k13q24sv1hxpmj2yhlsjlj-bash-5.1-p8/bin/bash", "propagatedBuildInputs=", "depsBuildBuildPropagated=", "NM_FOR_TARGET=nm", "CONFIG_SHELL=/nix/store/dfqckm5is1k13q24sv1hxpmj2yhlsjlj-bash-5.1-p8/bin/bash", "NIX_LDFLAGS=-rpath /nix/store/1549kfhbrsiad70d7mjvix9a80frxr0s-julia-1.9.0-beta2-patched/lib64 -rpath /nix/store/1549kfhbrsiad70d7mjvix9a80frxr0s-julia-1.9.0-beta2-patched/lib  -L/nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/lib -L/nix/store/2yy85x1bhwmynzmpr4n29caxpfm0bkk4-zlib-1.2.12/lib -L/nix/store/fmxkxd21yf4qklf4b07p8va881m0giq8-nghttp2-1.43.0-lib/lib -L/nix/store/ffa40clh365ccc4r47d4rkw5sz8i00gk-libidn-1.38/lib -L/nix/store/sxwqxjcqjxqa25chy8k4lg6di2lsy2zk-libkrb5-1.18/lib -L/nix/store/dmhq727zgz5zixq1vz22kqji5y1bgm5i-openssl-1.1.1o/lib -L/nix/store/drbyp7dgw4bi760zsyy6b7hzxp32zj28-libssh2-1.10.0/lib -L/nix/store/3gjnn0qidg12aqb9byjnp7f563mkv3s3-brotli-1.0.9-lib/lib -L/nix/store/1blwp2kqjg5xkk2gs3i3l8l36r1mhzqi-curl-7.79.1/lib -L/nix/store/v0vm93xrv2hf5wi175wskcgzbicfvbss-python3-3.9.13/lib -L/nix/store/2yy85x1bhwmynzmpr4n29caxpfm0bkk4-zlib-1.2.12/lib -L/nix/store/fmxkxd21yf4qklf4b07p8va881m0giq8-nghttp2-1.43.0-lib/lib -L/nix/store/ffa40clh365ccc4r47d4rkw5sz8i00gk-libidn-1.38/lib -L/nix/store/sxwqxjcqjxqa25chy8k4lg6di2lsy2zk-libkrb5-1.18/lib -L/nix/store/dmhq727zgz5zixq1vz22kqji5y1bgm5i-openssl-1.1.1o/lib -L/nix/store/drbyp7dgw4bi760zsyy6b7hzxp32zj28-libssh2-1.10.0/lib -L/nix/store/3gjnn0qidg12aqb9byjnp7f563mkv3s3-brotli-1.0.9-lib/lib -L/nix/store/1blwp2kqjg5xkk2gs3i3l8l36r1mhzqi-curl-7.79.1/lib", "PKG_CONFIG_PATH=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib/pkgconfig", "NIX_HARDENING_ENABLE=fortify stackprotector pic strictoverflow format relro bindnow", "NIX_ENFORCE_PURITY=1", "propagatedNativeBuildInputs=", "stdenv=/nix/store/b6ak2wii6k6j7m606c67rwyng1m7whnx-stdenv-linux", "MAKE_TERMOUT=/dev/pts/0", "JULIA_CPU_TARGET=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)", "USE_BINARYBUILDER=1", "OPENBLAS_MAIN_FREE=1", "LLVM_LDFLAGS=-lLLVM"]): no such file or directory (ENOENT)
Stacktrace:
  [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Vector{Union{RawFD, IO}})
    @ Base ./process.jl:128
  [2] #760
    @ ./process.jl:139 [inlined]
  [3] setup_stdios(f::Base.var"#760#761"{Cmd}, stdios::Vector{Union{RawFD, IO}})
    @ Base ./process.jl:223
  [4] _spawn
    @ ./process.jl:138 [inlined]
  [5] run(::Cmd, ::Base.DevNull, ::Vararg{Any}; wait::Bool)
    @ Base ./process.jl:479
  [6] run
    @ ./process.jl:477 [inlined]
  [7] link_image (repeats 2 times)
    @ ./linking.jl:144 [inlined]
  [8] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2055
  [9] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:2016
 [10] top-level scope
    @ none:3
ERROR: LoadError: failed process: Process(`/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/bin/julia -O0 --sysimage /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib/julia/sys.ji --trace-compile=/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/jl_sdFMzW/jl_V25encV07W --startup-file=no -Cnative -e 'pushfirst!(DEPOT_PATH, "/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/jl_sdFMzW");
Base.PRECOMPILE_TRACE_COMPILE[] = "/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/jl_sdFMzW/jl_dkN4rGzsFc";
Base.compilecache(Base.PkgId("__PackagePrecompilationStatementModule"), "/tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/jl_sdFMzW/__PackagePrecompilationStatementModule/src/__PackagePrecompilationStatementModule.jl")
# NOTE: these were moved to the end of Base.jl. TODO: move back here.
# # Used by Revise & its dependencies
# while true  # force inference
# delete!(push!(Set{Module}(), Base), Main)
# m = first(methods(+))
# delete!(push!(Set{Method}(), m), m)
# empty!(Set())
# push!(push!(Set{Union{GlobalRef,Symbol}}(), :two), GlobalRef(Base, :two))
# (setindex!(Dict{String,Base.PkgId}(), Base.PkgId(Base), "file.jl"))["file.jl"]
# (setindex!(Dict{Symbol,Vector{Int}}(), [1], :two))[:two]
# (setindex!(Dict{Base.PkgId,String}(), "file.jl", Base.PkgId(Base)))[Base.PkgId(Base)]
# (setindex!(Dict{Union{GlobalRef,Symbol}, Vector{Int}}(), [1], :two))[:two]
# (setindex!(IdDict{Type, Union{Missing, Vector{Tuple{LineNumberNode, Expr}}}}(), missing, Int))[Int]
# Dict{Symbol, Union{Nothing, Bool, Symbol}}(:one => false)[:one]
# Dict(Base => [:(1+1)])[Base]
# Dict(:one => [1])[:one]
# Dict("abc" => Set())["abc"]
# pushfirst!([], sum)
# get(Base.pkgorigins, Base.PkgId(Base), nothing)
# sort!([1,2,3])
# unique!([1,2,3])
# cumsum([1,2,3])
# append!(Int[], BitSet())
# isempty(BitSet())
# delete!(BitSet([1,2]), 3)
# deleteat!(Int32[1,2,3], [1,3])
# deleteat!(Any[1,2,3], [1,3])
# Core.svec(1, 2) == Core.svec(3, 4)
# # copy(Core.Compiler.retrieve_code_info(Core.Compiler.specialize_method(which(+, (Int, Int)), [Int, Int], Core.svec())))
# any(t->t[1].line > 1, [(LineNumberNode(2,:none),:(1+1))])
# break   # end force inference
# end
using Artifacts, Base.BinaryPlatforms, Libdl
artifacts_toml = abspath(joinpath(Sys.STDLIB, "Artifacts", "test", "Artifacts.toml"))
artifact_hash("HelloWorldC", artifacts_toml)
oldpwd = pwd(); cd(dirname(artifacts_toml))
macroexpand(Main, :(@artifact_str("HelloWorldC")))
cd(oldpwd)
artifacts = Artifacts.load_artifacts_toml(artifacts_toml)
platforms = [Artifacts.unpack_platform(e, "HelloWorldC", artifacts_toml) for e in artifacts["HelloWorldC"]]
best_platform = select_platform(Dict(p => triplet(p) for p in platforms))
dlopen("libjulia", RTLD_LAZY | RTLD_DEEPBIND)

'`, ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:565 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:480
  [3] run
    @ ./process.jl:477 [inlined]
  [4] (::Main.anonymous.var"#1#5"{Set{String}, String})(prec_path::String)
    @ Main.anonymous /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/contrib/generate_precompile.jl:274
  [5] mktempdir(fn::Main.anonymous.var"#1#5"{Set{String}, String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:760
  [6] mktempdir (repeats 2 times)
    @ ./file.jl:756 [inlined]
  [7] generate_precompile_statements()
    @ Main.anonymous /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/contrib/generate_precompile.jl:256
  [8] top-level scope
    @ /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/contrib/generate_precompile.jl:431
  [9] eval(m::Module, e::Any)
    @ Core ./boot.jl:370
 [10] top-level scope
    @ /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/contrib/generate_precompile.jl:10
in expression starting at /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/contrib/generate_precompile.jl:7
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
make[1]: *** [sysimage.mk:89: /tmp/nix-build-julia-1.9.0-beta2-patched.drv-0/source/usr/lib/julia/sys-o.a] Error 1
make: *** [Makefile:90: julia-sysimg-release] Error 2
builder for '/nix/store/1czsjh5d3cwqjm3s2hvi5p33v4c9gwv3-julia-1.9.0-beta2-patched.drv' failed with exit code 2
NHDaly commented 1 year ago

CC: @staticfloat -- i think this is similar to other source tarballs issues we've had in the past?

giordano commented 1 year ago

https://github.com/JuliaLang/julia/issues/48057#issuecomment-1368249936 suggested to provide an option to use system linker.

Side question: do you know https://github.com/Mic92/nix-ld?

staticfloat commented 1 year ago

Yeah, I think the only reasonable fix for this on our end is to add a build option to use a system linker. Not having the interpreter in the standard place is a bridge too far for us to handle in our binaries, I think. ;)

vchuravy commented 1 year ago

Alternativly we could use LLD as a library...

NHDaly commented 1 year ago

I'm really clueless about this stuff. @rbvermaa / @kpamnany is this a good enough resolution for us?

rbvermaa commented 1 year ago

I believe we discussed this in a different issue/PR, where we concluded that using USE_SYSTEM_LLD=1 is sufficient for use in nix.

Edit: https://github.com/JuliaLang/julia/pull/48261#issuecomment-1382058672

sjkelly commented 1 year ago

If this is nix you may need to add a listing in the autopatchelf step for the BinaryBuilder artifacts. Or you can call it manually from the makefile like so:

https://github.com/sjkelly/julia/commit/1a216b37888aff4e8e99143d74ade24931731813

The above patch and the corresponding shell.nix have worked fine for me for source builds on NixOS.

kpamnany commented 1 year ago

We're able to proceed, so closing this.