Closed ronisbr closed 8 years ago
Seems like CMake is having trouble rewriting the RPATH? For some libraries we do that manually using patchelf. I guess we could do the same for MbedTLS, though it is surprising that no rpath is being set in the first place. Can you look in the full compile commands if that's just not being passed or if the compiler is just not adding it for some reason?
Where can I find this?
Maybe here:
/usr/bin/gcc -m64 -fPIC -Wall -Wextra -W -Wdeclaration-after-statement -Wwrite-strings -Wlogical-op -Wshadow -Wmissing-declarations -Wmissing-prototypes -O2 -shared -Wl,-soname,libmbedcrypto.so.0 -o libmbedcrypto.so.2.3.0 CMakeFiles/mbedcrypto.dir/aes.c.o CMakeFiles/mbedcrypto.dir/aesni.c.o CMakeFiles/mbedcrypto.dir/arc4.c.o CMakeFiles/mbedcrypto.dir/asn1parse.c.o CMakeFiles/mbedcrypto.dir/asn1write.c.o CMakeFiles/mbedcrypto.dir/base64.c.o CMakeFiles/mbedcrypto.dir/bignum.c.o CMakeFiles/mbedcrypto.dir/blowfish.c.o CMakeFiles/mbedcrypto.dir/camellia.c.o CMakeFiles/mbedcrypto.dir/ccm.c.o CMakeFiles/mbedcrypto.dir/cipher.c.o CMakeFiles/mbedcrypto.dir/cipher_wrap.c.o CMakeFiles/mbedcrypto.dir/ctr_drbg.c.o CMakeFiles/mbedcrypto.dir/des.c.o CMakeFiles/mbedcrypto.dir/dhm.c.o CMakeFiles/mbedcrypto.dir/ecdh.c.o CMakeFiles/mbedcrypto.dir/ecdsa.c.o CMakeFiles/mbedcrypto.dir/ecjpake.c.o CMakeFiles/mbedcrypto.dir/ecp.c.o CMakeFiles/mbedcrypto.dir/ecp_curves.c.o CMakeFiles/mbedcrypto.dir/entropy.c.o CMakeFiles/mbedcrypto.dir/entropy_poll.c.o CMakeFiles/mbedcrypto.dir/error.c.o CMakeFiles/mbedcrypto.dir/gcm.c.o CMakeFiles/mbedcrypto.dir/havege.c.o CMakeFiles/mbedcrypto.dir/hmac_drbg.c.o CMakeFiles/mbedcrypto.dir/md.c.o CMakeFiles/mbedcrypto.dir/md2.c.o CMakeFiles/mbedcrypto.dir/md4.c.o CMakeFiles/mbedcrypto.dir/md5.c.o CMakeFiles/mbedcrypto.dir/md_wrap.c.o CMakeFiles/mbedcrypto.dir/memory_buffer_alloc.c.o CMakeFiles/mbedcrypto.dir/oid.c.o CMakeFiles/mbedcrypto.dir/padlock.c.o CMakeFiles/mbedcrypto.dir/pem.c.o CMakeFiles/mbedcrypto.dir/pk.c.o CMakeFiles/mbedcrypto.dir/pk_wrap.c.o CMakeFiles/mbedcrypto.dir/pkcs12.c.o CMakeFiles/mbedcrypto.dir/pkcs5.c.o CMakeFiles/mbedcrypto.dir/pkparse.c.o CMakeFiles/mbedcrypto.dir/pkwrite.c.o CMakeFiles/mbedcrypto.dir/platform.c.o CMakeFiles/mbedcrypto.dir/ripemd160.c.o CMakeFiles/mbedcrypto.dir/rsa.c.o CMakeFiles/mbedcrypto.dir/sha1.c.o CMakeFiles/mbedcrypto.dir/sha256.c.o CMakeFiles/mbedcrypto.dir/sha512.c.o CMakeFiles/mbedcrypto.dir/threading.c.o CMakeFiles/mbedcrypto.dir/timing.c.o CMakeFiles/mbedcrypto.dir/version.c.o CMakeFiles/mbedcrypto.dir/version_features.c.o CMakeFiles/mbedcrypto.dir/xtea.c.o -lz -Wl,-rpath,:::::::
Yep -Wl,-rpath,:::::::
, so it looks like it somehow messes up computing what the RPATH should be.
This patch allowed me to compile mbedtls:
Index: julia-0.5.0-rc1/deps/mbedtls.mk
===================================================================
--- julia-0.5.0-rc1.orig/deps/mbedtls.mk
+++ julia-0.5.0-rc1/deps/mbedtls.mk
@@ -23,7 +23,7 @@ MBEDTLS_OPTS += -DENABLE_ZLIB_SUPPORT=ON
endif
ifeq ($(OS),Linux)
-MBEDTLS_OPTS += -DCMAKE_INSTALL_RPATH="\$$ORIGIN"
+MBEDTLS_OPTS += -DCMAKE_SKIP_BUILD_RPATH=FALSE -DCMAKE_BUILD_WITH_INSTALL_RPATH=FALSE -DCMAKE_INSTALL_RPATH="\$$ORIGIN"
endif
$(SRCDIR)/srccache/$(MBEDTLS_SRC).tgz: | $(SRCDIR)/srccache
However, I have no idea if this is the right approach.
Does the resulting binary have an RPATH set (~/julia/usr/tools/patchelf --print-rpath
I need to check this in which file?
If you mean julia
executable, then:
patchelf --print-rpath ./julia-0.5-0.5.0.rc1-0.x86_64/usr/bin/julia 17:29:12
$ORIGIN/../lib64/julia:$ORIGIN/../lib64
I think Keno meant the mbedcrypto library. What distro and cmake version is this? If you're building a package against system copies of libgit2, you might not need mbedtls at all if the libgit2 you're building against supports https and proxies.
@tkelman, I'm pretty sure that libgit2
in openSUSE has support to https and proxies. Hence, what should I do? Set USE_SYSTEM_MBEDTLS=1 and not install mbedtls
?
I don't know whether USE_SYSTEM_MBEDTLS=1
by itself will work right now. USE_SYSTEM_LIBGIT2=1
if you have a new enough version should be able to avoid building any of mbedtls, libssh2, or libcurl, but the makefile logic may need adjusting to make that actually happen correctly.
Without USE_SYSTEM_MBEDTLS=1
, the build process tries to download the tarball.
Good! With USE_SYSTEM_MBEDTLS=1
julia builds fine. I'll send to build farm, and when it is completed, I will run the tests. Thanks for the help @tkelman and @Keno
The runtest.jl
fails:
# julia runtests.jl
From worker 7: * linalg/special in 4.15 seconds, maxrss 215.82 MB
From worker 6: * linalg/schur in 15.78 seconds, maxrss 259.24 MB
From worker 7: * linalg/svd in 32.70 seconds, maxrss 295.14 MB
From worker 6: * linalg/lapack in 26.38 seconds, maxrss 298.28 MB
From worker 9: * linalg/bunchkaufman in 42.31 seconds, maxrss 260.75 MB
From worker 4: * linalg/dense in 47.39 seconds, maxrss 316.79 MB
From worker 8: * linalg/eigen in 50.50 seconds, maxrss 298.29 MB
From worker 4: * linalg/pinv in 4.94 seconds, maxrss 405.73 MB
From worker 8: * linalg/givens in 3.63 seconds, maxrss 302.54 MB
From worker 7: * linalg/tridiag in 21.90 seconds, maxrss 321.18 MB
From worker 6: * linalg/bidiag in 25.34 seconds, maxrss 355.48 MB
From worker 3: * linalg/qr in 71.71 seconds, maxrss 339.54 MB
From worker 7: * linalg/symmetric in 13.57 seconds, maxrss 343.98 MB
From worker 6: * linalg/generic in 5.46 seconds, maxrss 359.21 MB
From worker 6: * linalg/hessenberg in 0.64 seconds, maxrss 361.27 MB
From worker 3: * linalg/uniformscaling in 4.21 seconds, maxrss 347.07 MB
From worker 9: * linalg/diagonal in 37.84 seconds, maxrss 328.90 MB
WARNING: Method definition f(Tuple{Vararg{Int64, #N<:Any}}, AbstractArray{#T<:Any, #N<:Any}) in module Main at /usr/share/julia/test/core.jl:706 overwritten at /usr/share/julia/test/core.jl:712.
From worker 4: * linalg/cholesky in 31.72 seconds, maxrss 417.71 MB
From worker 4: * inference in 0.45 seconds, maxrss 417.81 MB
From worker 4: * keywordargs in 0.98 seconds, maxrss 417.97 MB
WARNING: Method definition f() in module JLCall14301 at /usr/share/julia/test/core.jl:3529 overwritten at /usr/share/julia/test/core.jl:3539.
From worker 8: * linalg/lu in 36.62 seconds, maxrss 359.25 MB
From worker 6: * linalg/arnoldi in 22.24 seconds, maxrss 424.84 MB
From worker 6: * char in 0.55 seconds, maxrss 426.89 MB
From worker 9: * core in 19.39 seconds, maxrss 350.79 MB
From worker 9: * triplequote in 0.01 seconds, maxrss 350.79 MB
From worker 8: * printf in 11.04 seconds, maxrss 365.68 MB
From worker 9: * unicode in 3.38 seconds, maxrss 366.82 MB
From worker 6: * string in 7.74 seconds, maxrss 432.71 MB
From worker 6: * hashing in 3.71 seconds, maxrss 435.04 MB
From worker 5: * linalg/matmul in 108.17 seconds, maxrss 310.10 MB
From worker 6: * iobuffer in 0.53 seconds, maxrss 435.07 MB
From worker 5: * staged in 0.62 seconds, maxrss 310.67 MB
From worker 9: * dict in 7.28 seconds, maxrss 377.04 MB
From worker 9: * tuple in 1.18 seconds, maxrss 380.23 MB
WARNING: could not attach metadata for @simd loop.
From worker 9: * reduce in 3.01 seconds, maxrss 402.70 MB
From worker 9: * reducedim in 10.57 seconds, maxrss 419.09 MB
From worker 8: * dates in 26.04 seconds, maxrss 431.94 MB
From worker 4: * numbers in 43.74 seconds, maxrss 516.32 MB
From worker 4: * intfuncs in 0.43 seconds, maxrss 516.32 MB
From worker 4: * simdloop in 0.91 seconds, maxrss 516.32 MB
From worker 4: * vecelement in 1.29 seconds, maxrss 516.32 MB
From worker 9: * random in 8.34 seconds, maxrss 452.50 MB
From worker 7: * linalg/lq in 62.31 seconds, maxrss 398.43 MB
From worker 6: * offsetarray in 31.02 seconds, maxrss 490.89 MB
From worker 6: * copy in 1.00 seconds, maxrss 491.06 MB
From worker 5: * arrayops in 34.68 seconds, maxrss 473.50 MB
From worker 5: * fastmath in 3.31 seconds, maxrss 473.50 MB
From worker 4: * blas in 23.90 seconds, maxrss 546.77 MB
From worker 4: * operators in 0.41 seconds, maxrss 546.86 MB
From worker 4: * path in 3.07 seconds, maxrss 551.50 MB
From worker 4: * ccall in 3.40 seconds, maxrss 553.09 MB
From worker 4: * parse in 1.39 seconds, maxrss 553.40 MB
From worker 4: * loading in 0.05 seconds, maxrss 553.48 MB
From worker 4: * bigint in 1.01 seconds, maxrss 553.80 MB
From worker 4: * bigfloat in 0.03 seconds, maxrss 553.81 MB
From worker 5: * functional in 20.05 seconds, maxrss 473.50 MB
From worker 6: * math in 32.17 seconds, maxrss 528.79 MB
From worker 6: * spawn [stdio passthrough ok]
From worker 5: * statistics in 6.70 seconds, maxrss 473.50 MB
From worker 5: * backtrace From worker 8: * abstractarray in 47.32 seconds, maxrss 489.87 MB
From worker 8: * priorityqueue in 1.02 seconds, maxrss 505.51 MB
From worker 6: in 15.94 seconds, maxrss 534.40 MB
WARNING: file permissions tests skipped due to running tests as root (not recommended)
WARNING: file permissions tests skipped due to running tests as root (not recommended)
From worker 10: * read in 12.55 seconds, maxrss 293.01 MB
From worker 10: * version in 0.74 seconds, maxrss 294.03 MB
From worker 10: * resolve in 2.58 seconds, maxrss 305.36 MB
From worker 4: * sorting in 30.02 seconds, maxrss 613.45 MB
From worker 10: * pollfd in 3.41 seconds, maxrss 307.42 MB
From worker 6: * mmap in 7.84 seconds, maxrss 534.42 MB
From worker 4: * mpfr in 1.46 seconds, maxrss 613.73 MB
From worker 4: * socket in 2.40 seconds, maxrss 614.71 MB
From worker 4: * floatapprox in 0.19 seconds, maxrss 615.35 MB
From worker 6: * complex in 4.82 seconds, maxrss 534.42 MB
From worker 4: * datafmt in 4.94 seconds, maxrss 625.24 MB
From worker 4: * regex in 0.31 seconds, maxrss 626.02 MB
From worker 6: * reflection in 3.51 seconds, maxrss 534.42 MB
From worker 6: * combinatorics in 0.28 seconds, maxrss 534.42 MB
From worker 4: * float16 in 0.63 seconds, maxrss 627.14 MB
From worker 6: * sysinfo in 0.57 seconds, maxrss 534.42 MB
From worker 4: * rounding in 0.46 seconds, maxrss 628.78 MB
From worker 4: * mod2pi in 0.11 seconds, maxrss 628.88 MB
From worker 4: * euler in 0.54 seconds, maxrss 638.28 MB
From worker 8: * file in 36.00 seconds, maxrss 516.92 MB
From worker 8: * lineedit in 2.09 seconds, maxrss 516.92 MB
From worker 10: * broadcast in 19.63 seconds, maxrss 325.60 MB
From worker 8: * replcompletions in 2.59 seconds, maxrss 516.92 MB
From worker 4: * show in 12.42 seconds, maxrss 669.59 MB
From worker 10: * repl in 3.33 seconds, maxrss 330.26 MB
From worker 8: * replutil Test Failed
From worker 8: Expression: contains(err_str,"Cannot raise an integer x to a negative power -n")
From worker 6: * ranges in 14.79 seconds, maxrss 536.28 MB
From worker 6: * goto in 0.03 seconds, maxrss 536.28 MB
From worker 6: * llvmcall in 0.09 seconds, maxrss 536.40 MB
From worker 4: * sets in 2.06 seconds, maxrss 671.77 MB
From worker 6: * grisu in 1.65 seconds, maxrss 540.54 MB
From worker 6: * meta in 0.21 seconds, maxrss 542.66 MB
From worker 11: * stacktraces in 1.06 seconds, maxrss 215.05 MB
From worker 10: * test in 5.37 seconds, maxrss 340.32 MB
From worker 6: * profile in 2.69 seconds, maxrss 550.80 MB
From worker 2: * linalg/triangular in 225.88 seconds, maxrss 681.95 MB
From worker 2: * base64 in 0.20 seconds, maxrss 682.11 MB
From worker 10: * docs in 2.27 seconds, maxrss 354.54 MB
From worker 4: * nullable in 7.07 seconds, maxrss 676.41 MB
From worker 6: * markdown in 2.84 seconds, maxrss 551.36 MB
Warning: threaded loop executed in order
From worker 6: * enums in 1.36 seconds, maxrss 554.39 MB
From worker 2: * serialize in 3.96 seconds, maxrss 686.52 MB
From worker 2: * i18n in 0.01 seconds, maxrss 686.52 MB
From worker 4: * threads in 2.53 seconds, maxrss 685.72 MB
From worker 4: * libdl in 0.38 seconds, maxrss 685.72 MB
From worker 2: * workspace in 0.53 seconds, maxrss 686.52 MB
From worker 11: * libgit2 in 8.88 seconds, maxrss 247.88 MB
From worker 4: * int in 1.86 seconds, maxrss 685.72 MB
From worker 11: * intset in 0.41 seconds, maxrss 247.98 MB
From worker 10: * misc in 7.13 seconds, maxrss 379.36 MB
From worker 10: * inline in 0.13 seconds, maxrss 379.37 MB
From worker 4: * floatfuncs in 1.83 seconds, maxrss 685.72 MB
From worker 4: * error in 0.64 seconds, maxrss 685.72 MB
From worker 10: * boundscheck in 1.66 seconds, maxrss 379.39 MB
From worker 10: * cartesian in 0.01 seconds, maxrss 379.39 MB
WARNING: Method definition ambig(Any, Integer) in module Main at /usr/share/julia/test/ambiguous.jl:7 overwritten at /usr/share/julia/test/ambiguous.jl:79.
From worker 4: * ambiguous Skipping Base.<|
From worker 4: Skipping Base.>:
From worker 4: Skipping Base.active_repl
From worker 4: Skipping Base.active_repl_backend
From worker 4: in 0.91 seconds, maxrss 685.72 MB
From worker 2: * checked in 7.81 seconds, maxrss 702.62 MB
From worker 7: * bitarray in 104.68 seconds, maxrss 444.86 MB
From worker 4: * dsp in 7.33 seconds, maxrss 689.88 MB
From worker 10: * fft in 13.05 seconds, maxrss 403.82 MB
From worker 11: * parallel in 50.31 seconds, maxrss 248.05 MB
From worker 9: * sparse in 176.08 seconds, maxrss 715.23 MB
From worker 3: * subarray in 247.49 seconds, maxrss 814.52 MB
From worker 6: * cmdlineargs Test Failed
From worker 6: Expression: contains(bt,"in include_from_node1")
Exception running test backtrace :
On worker 5:
LoadError: BoundsError: attempt to access svec()
at index [1]
in macro expansion; at /usr/share/julia/test/backtrace.jl:7 [inlined]
in anonymous at ./<missing>:?
in macro expansion at ./util.jl:226 [inlined]
in runtests at /usr/share/julia/test/testdefs.jl:7
in #16 at /usr/share/julia/test/runtests.jl:44
in #503 at ./multi.jl:1410
in run_work_thunk at ./multi.jl:996
in macro expansion at ./multi.jl:1410 [inlined]
in #502 at ./event.jl:46
while loading /usr/share/julia/test/backtrace.jl, in expression starting on line 5
Exception running test replutil :
On worker 8:
LoadError: There was an error during testing
in record at ./test.jl:397
in do_test at ./test.jl:281
in macro expansion at ./util.jl:226 [inlined]
in runtests at /usr/share/julia/test/testdefs.jl:7
in #16 at /usr/share/julia/test/runtests.jl:44
in #503 at ./multi.jl:1410
in run_work_thunk at ./multi.jl:996
in macro expansion at ./multi.jl:1410 [inlined]
in #502 at ./event.jl:46
while loading /usr/share/julia/test/replutil.jl, in expression starting on line 261
Exception running test cmdlineargs :
On worker 6:
LoadError: There was an error during testing
in record at ./test.jl:397
in do_test at ./test.jl:281
in macro expansion; at /usr/share/julia/test/cmdlineargs.jl:299 [inlined]
in anonymous at ./<missing>:?
in macro expansion at ./util.jl:226 [inlined]
in runtests at /usr/share/julia/test/testdefs.jl:7
in #16 at /usr/share/julia/test/runtests.jl:44
in #503 at ./multi.jl:1410
in run_work_thunk at ./multi.jl:996
in macro expansion at ./multi.jl:1410 [inlined]
in #502 at ./event.jl:46
while loading /usr/share/julia/test/cmdlineargs.jl, in expression starting on line 296
ERROR: LoadError: Some tests exited with errors.
in (::##11#19)() at /usr/share/julia/test/runtests.jl:72
in cd(::##11#19, ::String) at ./file.jl:59
while loading /usr/share/julia/test/runtests.jl, in expression starting on line 26
But I think it is not related to this problem, right?
I opened a new bug #17941 I think we can close this issue since I can now build julia, right?
What distro and cmake version is this?
And what was rpath on the mbedcrypto library? We would like to resolve the issue you were having with source builds if it's reproducible with a standard-configuration make
in your environment.
@tkelman it turns out that mbedtls was not built. The source was not even unpacked. I have no idea what happened, but I could not make it build even if the patch I proposed. I think that the build process (openSUSE) added the libgit2 and julia skipped mbedtls. As of now, I cannot build mbedtls together with julia.
Are you setting anything in Make.user
or on the make
command line, or do you even have this problem in a plain git clone https://github.com/JuliaLang/julia && make -C julia
?
@tkelman actually I'm trying to make a package for openSUSE. Here is what I'm using for the make command line:
NO_GIT=1 \
USE_SYSTEM_LLVM=0 \
USE_SYSTEM_LIBUNWIND=1 \
USE_SYSTEM_READLINE=1 \
USE_SYSTEM_PCRE=1 \
USE_SYSTEM_LIBM=0 \
USE_SYSTEM_OPENLIBM=1 \
USE_SYSTEM_OPENSPECFUN=1 \
USE_SYSTEM_DSFMT=1 \
USE_SYSTEM_BLAS=1 \
LIBBLASNAME=libblas.so.3 \
USE_SYSTEM_LAPACK=1 \
LIBBLASNAME=liblapack.so.3 \
USE_SYSTEM_FFTW=1 \
LIBFFTWNAME=libfftw3_threads.so.3 \
LIBFFTWFNAME=libfftw3f_threads.so.3 \
USE_SYSTEM_GMP=1 \
USE_SYSTEM_MPFR=1 \
USE_SYSTEM_ARPACK=1 \
USE_SYSTEM_SUITESPARSE=1 \
USE_SYSTEM_ZLIB=1 \
USE_SYSTEM_GRISU=1 \
USE_SYSTEM_RMATH=0 \
USE_SYSTEM_LIBUV=0 \
USE_SYSTEM_UTF8PROC=0 \
USE_SYSTEM_PATCHELF=1 \
USE_SYSTEM_LIBGIT2=1 \
USE_SYSTEM_MBEDTLS=1 \
USE_SYSTEM_LIBSSH2=1 \
USE_MKL=0 \
USE_BLAS64=0 \
USE_LLVM_SHLIB=0 \
VERBOSE=1 \
USECLANG=0
For the records, if USE_SYSTEM_MBEDTLS=1
, then I can build julia even not installing mbedtls at all.
In that case I'll close this as a duplicate of https://github.com/JuliaLang/julia/issues/17884. That kind of information would be best to include with the initial bug report next time.
@tkelman sorry, I didn't understand. I just discovered that I can build julia without mbedtls after I opened this bug report.
Here is what I'm using for the make command line:
That list would have been valuable to include when you initially opened the issue is all, so we can have as much information as possible to understand what's going on and correctly diagnose the problem.
Ah! Ok :) sorry
Hi guys!
I'm trying to pack julia 0.5.0-rc1, but the mbedtls is failing with this message:
Can anyone help me?