JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.73k stars 5.48k forks source link

BUILD_LLVM_CLANG=1 fails in Make.user #19517

Closed lascott closed 7 years ago

lascott commented 7 years ago

The following Make.user fails to build:

MARCH=native
BUILD_LLVM_CLANG=1

Commenting out the BUILD_LLVM_CLANG line works just fine. Any idea what to add to ensure Clang is built? @vchuravy @iamed2 @Keno , I have some time to devote at the moment to bootstrapping a NAG.jl using Clang.jl but then the holidays will hit. Any advice would be most appreciated. Thanks!

I am able to install and run clang on its own using 3.8.0. (See Clang.jl issue no. 142 for details)

But when I ask Make.user to BUILD_LLVM_CLANG=1, results in exact same error as #18926 on october 17 comment

/bin/mkdir -p '/mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib'
 /bin/bash ./libtool   --mode=install /usr/bin/install -c   libuv.la '/mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib'
libtool: install: /usr/bin/install -c .libs/libuv.so.1.0.0 /mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib/libuv.so.1.0.0
libtool: install: (cd /mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib && { ln -s -f libuv.so.1.0.0 libuv.so.1 || { rm -f libuv.so.1 && ln -s libuv.so.1.0.0 libuv.so.1; }; })
libtool: install: (cd /mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib && { ln -s -f libuv.so.1.0.0 libuv.so || { rm -f libuv.so && ln -s libuv.so.1.0.0 libuv.so; }; })
libtool: install: /usr/bin/install -c .libs/libuv.lai /mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib/libuv.la
libtool: install: /usr/bin/install -c .libs/libuv.a /mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib/libuv.a
libtool: install: chmod 644 /mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib/libuv.a
libtool: install: ranlib /mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib/libuv.a
libtool: warning: remember to run 'libtool --finish /mypath/julia/dev/usr/lib'
 /bin/mkdir -p '/mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/include'
 /usr/bin/install -c -m 644 /mypath/julia/dev/deps/srccache/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/include/uv.h /mypath/julia/dev/deps/srccache/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/include/uv-errno.h /mypath/julia/dev/deps/srccache/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/include/uv-threadpool.h /mypath/julia/dev/deps/srccache/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/include/uv-version.h /mypath/julia/dev/deps/srccache/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/include/uv-unix.h /mypath/julia/dev/deps/srccache/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/include/uv-linux.h '/mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/include'
 /bin/mkdir -p '/mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib/pkgconfig'
 /usr/bin/install -c -m 644 libuv.pc '/mypath/julia/dev/usr-staging/libuv-8d5131b6c1595920dd30644cd1435b4f344b46c8/mypath/julia/dev/usr/lib/pkgconfig'
can't find file to patch at input line 25
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|From efecb2c285bd444b6def43ac62e5f0278df387eb Mon Sep 17 00:00:00 2001
|From: Keno Fischer <kfischer@college.harvard.edu>
|Date: Mon, 5 Oct 2015 22:24:12 +0000
|Subject: [PATCH] [compiler-rt] Properly detect lack of available system
| libraries for arch in clang_darwin.mk
|
|Summary: This is the Makefile analog of r247833, except that the test also had to be changed such that clang actually attempts to link the program as opposed to just building it. Because of that change, I also switched the order to checking for ld/clang architecture support, because now lack of ld support would make the clang check fail. This fixes PR24776.
|
|Reviewers: beanz
|
|Subscribers: llvm-commits
|
|Differential Revision: http://reviews.llvm.org/D13425
|
|git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@249358 91177308-0d34-0410-b5e6-96231b3b80d8
|---
| make/platform/clang_darwin.mk           | 16 ++++++++--------
| make/platform/clang_darwin_test_input.c |  9 +++++++++
| 2 files changed, 17 insertions(+), 8 deletions(-)
|
|diff --git a/make/platform/clang_darwin.mk b/make/platform/clang_darwin.mk
|index 3a034b8..dff2694 100644
|--- a/projects/compiler-rt/make/platform/clang_darwin.mk
|+++ b/projects/compiler-rt/make/platform/clang_darwin.mk
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
1 out of 1 hunk ignored
can't find file to patch at input line 61
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/make/platform/clang_darwin_test_input.c b/make/platform/clang_darwin_test_input.c
|index b7074b8..b406a28 100644
|--- a/projects/compiler-rt/make/platform/clang_darwin_test_input.c
|+++ b/projects/compiler-rt/make/platform/clang_darwin_test_input.c
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
1 out of 1 hunk ignored
make[1]: *** [/mypath/julia/dev/deps/srccache/llvm-3.7.1/compiler-rt-3.7.1.patch-applied] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [julia-deps] Error 2
iamed2 commented 7 years ago

Have you tried with a fresh clone of the Julia repo?

lascott commented 7 years ago

I am using the 6 day old commit, as it was the most recent that successfully built. I do

# git clone git://github.com/JuliaLang/julia.git
cp -a julia dev
cd dev
git checkout dae92d3a0da2df39077e3f243e1336acf572f818
make -j 2  2>&1 | tee build.log

with Make.user MARCH=native BUILD_LLVM_CLANG=1

results in the following. Can I ask what version of gcc you are using?

llvm[6]: Compiling ParseAST.cpp for Release build
llvm[6]: Compiling PPCaching.cpp for Release build
llvm[6]: Compiling PPCallbacks.cpp for Release build
llvm[6]: Compiling ParseCXXInlineMethods.cpp for Release build
llvm[6]: Compiling PPConditionalDirectiveRecord.cpp for Release build
llvm[6]: Compiling PPDirectives.cpp for Release build
llvm[6]: Compiling ParseDecl.cpp for Release build
llvm[6]: Compiling PPExpressions.cpp for Release build
llvm[6]: Compiling PPLexerChange.cpp for Release build
llvm[6]: Compiling PPMacroExpansion.cpp for Release build
llvm[6]: Compiling ParseDeclCXX.cpp for Release build
llvm[6]: Compiling PTHLexer.cpp for Release build
llvm[6]: Compiling Pragma.cpp for Release build
llvm[6]: Compiling ParseExpr.cpp for Release build
llvm[6]: Compiling PreprocessingRecord.cpp for Release build
llvm[6]: Compiling Preprocessor.cpp for Release build
llvm[6]: Compiling ParseExprCXX.cpp for Release build
llvm[6]: Compiling PreprocessorLexer.cpp for Release build
llvm[6]: Compiling ScratchBuffer.cpp for Release build
llvm[6]: Compiling TokenConcatenation.cpp for Release build
llvm[6]: Compiling ParseInit.cpp for Release build
llvm[6]: Compiling TokenLexer.cpp for Release build
llvm[6]: Building Release Archive Library libclangLex.a
llvm[6]: Compiling APValue.cpp for Release build
llvm[6]: Compiling ParseObjc.cpp for Release build
llvm[6]: Compiling ASTConsumer.cpp for Release build
llvm[6]: Compiling ASTContext.cpp for Release build
llvm[6]: Compiling ParseOpenMP.cpp for Release build
llvm[6]: Compiling ParsePragma.cpp for Release build
llvm[6]: Compiling ParseStmt.cpp for Release build
llvm[6]: Compiling ASTDiagnostic.cpp for Release build
llvm[6]: Compiling ParseStmtAsm.cpp for Release build
llvm[6]: Compiling ASTDumper.cpp for Release build
In file included from /media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMap.h:17:0,
                 from /media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/tools/clang/lib/Parse/../../include/clang/Basic/Diagnostic.h:23,
                 from /media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/tools/clang/lib/Parse/../../include/clang/Lex/Preprocessor.h:19,
                 from /media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/tools/clang/lib/Parse/../../include/clang/Parse/Parser.h:21,
                 from /media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/tools/clang/lib/Parse/ParseStmtAsm.cpp:14:
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMapInfo.h: In instantiation of ‘static T* llvm::DenseMapInfo<T*>::getEmptyKey() [with T = const clang::ObjCContainerDecl]’:
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMap.h:344:34:   required from ‘static const KeyT llvm::DenseMapBase<DerivedT, KeyT, ValueT, KeyInfoT, BucketT>::getEmptyKey() [with DerivedT = llvm::DenseMap<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>; KeyT = const clang::ObjCContainerDecl*; ValueT = const clang::ASTRecordLayout*; KeyInfoT = llvm::DenseMapInfo<const clang::ObjCContainerDecl*>; BucketT = llvm::detail::DenseMapPair<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>]’
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMap.h:480:39:   required from ‘bool llvm::DenseMapBase<DerivedT, KeyT, ValueT, KeyInfoT, BucketT>::LookupBucketFor(const LookupKeyT&, const BucketT*&) const [with LookupKeyT = const clang::ObjCContainerDecl*; DerivedT = llvm::DenseMap<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>; KeyT = const clang::ObjCContainerDecl*; ValueT = const clang::ASTRecordLayout*; KeyInfoT = llvm::DenseMapInfo<const clang::ObjCContainerDecl*>; BucketT = llvm::detail::DenseMapPair<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>]’
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMap.h:522:46:   required from ‘bool llvm::DenseMapBase<DerivedT, KeyT, ValueT, KeyInfoT, BucketT>::LookupBucketFor(const LookupKeyT&, BucketT*&) [with LookupKeyT = const clang::ObjCContainerDecl*; DerivedT = llvm::DenseMap<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>; KeyT = const clang::ObjCContainerDecl*; ValueT = const clang::ASTRecordLayout*; KeyInfoT = llvm::DenseMapInfo<const clang::ObjCContainerDecl*>; BucketT = llvm::detail::DenseMapPair<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>]’
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMap.h:227:39:   required from ‘llvm::DenseMapBase<DerivedT, KeyT, ValueT, KeyInfoT, BucketT>::value_type& llvm::DenseMapBase<DerivedT, KeyT, ValueT, KeyInfoT, BucketT>::FindAndConstruct(const KeyT&) [with DerivedT = llvm::DenseMap<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>; KeyT = const clang::ObjCContainerDecl*; ValueT = const clang::ASTRecordLayout*; KeyInfoT = llvm::DenseMapInfo<const clang::ObjCContainerDecl*>; BucketT = llvm::detail::DenseMapPair<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>; llvm::DenseMapBase<DerivedT, KeyT, ValueT, KeyInfoT, BucketT>::value_type = llvm::detail::DenseMapPair<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>]’
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMap.h:234:32:   required from ‘ValueT& llvm::DenseMapBase<DerivedT, KeyT, ValueT, KeyInfoT, BucketT>::operator[](const KeyT&) [with DerivedT = llvm::DenseMap<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>; KeyT = const clang::ObjCContainerDecl*; ValueT = const clang::ASTRecordLayout*; KeyInfoT = llvm::DenseMapInfo<const clang::ObjCContainerDecl*>; BucketT = llvm::detail::DenseMapPair<const clang::ObjCContainerDecl*, const clang::ASTRecordLayout*>]’
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/tools/clang/lib/Parse/../../include/clang/AST/ASTContext.h:2174:19:   required from here
/media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/include/llvm/ADT/DenseMapInfo.h:39:3: internal compiler error: Segmentation fault
   }
   ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
llvm[6]: Compiling ASTImporter.cpp for Release build
The bug is not reproducible, so it is likely a hardware or OS problem.
make[6]: *** [/media/louis/dtb/home/las/code/julia/devclang/deps/scratch/llvm-3.7.1/build_Release/tools/clang/lib/Parse/Release/ParseStmtAsm.o] Error 1
make[5]: *** [Parse/.makeall] Error 2
make[5]: *** Waiting for unfinished jobs....
llvm[6]: Compiling ASTTypeTraits.cpp for Release build
llvm[6]: Compiling AttrImpl.cpp for Release build
In file included from /media/louis/dtb/home/las/code/julia/devclang/deps/srccache/llvm-3.7.1/tools/clang/lib/AST/AttrImpl.cpp:21:0:
/media/louis/dtb/home/las/code/julia/devclang/deps/scratch/llvm-3.7.1/build_Release/tools/clang/lib/AST/../../include/clang/AST/AttrImpl.inc:7252:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
llvm[6]: Compiling CXXInheritance.cpp for Release build
llvm[6]: Compiling Comment.cpp for Release build
The bug is not reproducible, so it is likely a hardware or OS problem.
make[6]: *** [/media/louis/dtb/home/las/code/julia/devclang/deps/scratch/llvm-3.7.1/build_Release/tools/clang/lib/AST/Release/AttrImpl.o] Error 1
iamed2 commented 7 years ago

Personally? I use Apple's clang to build Julia.

Keno commented 7 years ago

Which OS is this on? Perhaps you ran out of memory?

lascott commented 7 years ago

Thanks Keno, watching memory usage while it compiles. Does not seem to go anywhere near full use. CPU use is high however. What are you compiling with?

Perusing the google, there are a number of hits to "internal compiler error: Segmentation fault llvm" A theme is to change gcc versions.

DISTRIB_CODENAME=rosa
DISTRIB_DESCRIPTION="Linux Mint 17.3 Rosa"
NAME="Ubuntu"
VERSION="14.04.4 LTS, Trusty Tahr"

gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

$ free -t -m
             total       used       free     shared    buffers     cached
Mem:         18054       9233       8821         97        466       6092
-/+ buffers/cache:       2674      15380
Swap:        18796       2743      16053
Total:       36851      11976      24874
lascott commented 7 years ago

Build succeeds with gcc5 I will leave open for Q&A before closing.

# To install newer versions of the compilers
#
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update

# installed in order  
# 4.9 FAIL. If memory serves, it stopped at building LAPACK, I think std=d99 was not set.
# 5.0 SUCCESS  added USE_SYSTEM_LAPACK=1 to Make.user

sudo apt-get install gcc-4.9 g++-4.9 gfortran-4.9  2>&1 | tee buildgcc.log
sudo apt-get install gcc-5 g++-5 gfortran-5  2>&1 | tee buildgcc.log

# https://github.com/scienceopen/pybashutils/blob/master/update-compilers.sh
source update-compilers.sh

cp -a julia devclang
cd devclang
git checkout dae92d3a0da2df39077e3f243e1336acf572f818
make -j 2  2>&1 | tee build.log

using Make.user:
MARCH=native
USE_SYSTEM_LAPACK=1
BUILD_LLVM_CLANG=1

Everything builds on make. In julia, all tests are successful. I expect earlier versions of the compiler should work, in particular 4.9. I was interested in getting it to work on running clang.jl so that is next.

louis@Neruda /mypath/julia/devclang $ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-dev.1324 (2016-11-28 15:34 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit dae92d3* (11 days old master)
|__/                   |  x86_64-linux-gnu

julia> Base.runtests("all",2)
        From worker 3:       * linalg/qr             maxrss  350.59 MB
        From worker 3:       * linalg/dense          maxrss  379.46 MB
        From worker 3:       * linalg/matmul         maxrss  445.50 MB
        From worker 3:       * linalg/schur          maxrss  450.61 MB
        From worker 3:       * linalg/special        maxrss  465.63 MB
        From worker 2:       * linalg/triangular     maxrss  601.18 MB
        From worker 3:       * linalg/eigen          maxrss  474.65 MB
        From worker 3:       * linalg/svd            maxrss  480.48 MB
        From worker 2:       * linalg/bunchkaufman   maxrss  636.55 MB
        From worker 3:       * linalg/lapack         maxrss  491.08 MB
        From worker 2:       * linalg/tridiag        maxrss  675.62 MB
        From worker 3:       * linalg/bidiag         maxrss  494.70 MB
        From worker 3:       * linalg/pinv           maxrss  579.59 MB
        From worker 3:       * linalg/givens         maxrss  582.66 MB
        From worker 2:       * linalg/diagonal       maxrss  690.14 MB
        From worker 3:       * linalg/cholesky       maxrss  589.88 MB
        From worker 3:       * linalg/symmetric      maxrss  594.74 MB
        From worker 3:       * linalg/generic        maxrss  596.23 MB
        From worker 2:       * linalg/lu             maxrss  725.86 MB
        From worker 3:       * linalg/uniformscaling maxrss  599.59 MB
        From worker 3:       * linalg/hessenberg     maxrss  599.59 MB
        From worker 2:       * linalg/lq             maxrss  805.55 MB
        From worker 3:       * linalg/arnoldi        maxrss  600.90 MB
        From worker 3:       * sparse/sparse         maxrss  650.15 MB
        From worker 3:       * sparse/sparsevector   maxrss  706.00 MB
        From worker 3:       * sparse/umfpack        maxrss  711.22 MB
        From worker 3:       * sparse/cholmod        maxrss  749.46 MB
        From worker 3:       * sparse/spqr           maxrss  749.46 MB
        From worker 3:       * strings/basic         maxrss  749.46 MB
        From worker 3:       * strings/search        maxrss  749.46 MB
        From worker 3:       * strings/util          maxrss  749.46 MB
        From worker 3:       * strings/io            maxrss  749.46 MB
        From worker 3:       * strings/types         maxrss  749.46 MB
        From worker 3:       * unicode/UnicodeError  maxrss  749.46 MB
        From worker 3:       * unicode/utf8proc      maxrss  749.46 MB
        From worker 2:       * subarray              maxrss 1139.21 MB
        From worker 3:       * unicode/utf8          maxrss  795.05 MB
        From worker 3:       * dates/adjusters       maxrss  811.21 MB
        From worker 3:       * dates/query           maxrss  813.27 MB
        From worker 3:       * dates/periods         maxrss  813.80 MB
        From worker 2:       * dates/accessors       maxrss 1797.75 MB
        From worker 2:       * dates/rounding        maxrss 1924.74 MB
        From worker 2:       * dates/types           maxrss 1934.20 MB
        From worker 2:       * dates/io              maxrss 1985.78 MB
        From worker 2:       * dates/arithmetic      maxrss 2020.57 MB
        From worker 2:       * dates/conversions     maxrss 2033.95 MB
WARNING: Method definition f(Tuple{Vararg{Int64, #N<:Any}}, AbstractArray{#T<:Any, #N<:Any}) in module TestMain_core at /mypath/julia/dev/test/core.jl:729 overwritten at /mypath/julia/dev/test/core.jl:735.
        From worker 3:       * dates/ranges          maxrss  849.48 MB
        From worker 3:       * inference             maxrss  853.53 MB
        From worker 3:       * keywordargs           maxrss  853.63 MB
WARNING: Method definition f() in module JLCall14301 at /mypath/julia/dev/test/core.jl:3559 overwritten at /mypath/julia/dev/test/core.jl:3569.
WARNING: static parameter T does not occur in signature for bad_tvars at /mypath/julia/dev/test/core.jl:4677.
The method will not be callable.
        From worker 2:       * core                  maxrss 2526.68 MB
        From worker 2:       * printf                maxrss 2526.68 MB
        From worker 2:       * char                  maxrss 2526.68 MB
        From worker 2:       * triplequote           maxrss 2526.68 MB
        From worker 2:       * dict                  maxrss 2526.68 MB
        From worker 3:       * numbers               maxrss 1073.23 MB
        From worker 3:       * iobuffer              maxrss 1077.29 MB
        From worker 2:       * hashing               maxrss 2526.68 MB
        From worker 3:       * staged                maxrss 1079.45 MB
        From worker 2:       * offsetarray           maxrss 2526.68 MB
        From worker 2:       * tuple                 maxrss 2526.68 MB
        From worker 2:       * reduce                maxrss 2526.68 MB
        From worker 3:       * arrayops              maxrss 1292.36 MB
        From worker 3:       * random                maxrss 1292.36 MB
        From worker 2:       * reducedim             maxrss 2526.68 MB
        From worker 2:       * intfuncs              maxrss 2526.68 MB
        From worker 2:       * simdloop              maxrss 2526.68 MB
        From worker 2:       * vecelement            maxrss 2526.68 MB
        From worker 2:       * blas                  maxrss 2526.68 MB
        From worker 3:       * abstractarray         maxrss 1292.36 MB
        From worker 3:       * copy                  maxrss 1292.36 MB
        From worker 3:       * math                  maxrss 1356.03 MB
        From worker 3:       * fastmath              maxrss 1356.34 MB
        From worker 3:       * functional            maxrss 1356.71 MB
        From worker 3:       * iterators             maxrss 1375.49 MB
        From worker 3:       * operators             maxrss 1379.16 MB
        From worker 3:       * path                  maxrss 1379.16 MB
        From worker 3:       * ccall                 maxrss 1379.16 MB
        From worker 3:       * parse                 maxrss 1379.16 MB
        From worker 3:       * loading               maxrss 1379.16 MB
        From worker 3:       * bigint                maxrss 1379.16 MB
        From worker 3:       * bigfloat              maxrss 1379.16 MB
        From worker 3:       * sorting               maxrss 1395.66 MB
        From worker 3:       * statistics            maxrss 1403.16 MB
        From worker 3:       * spawn                       [stdio passthrough ok]
        From worker 3:   maxrss 1416.60 MB
        From worker 3:       * backtrace             maxrss 1421.23 MB
        From worker 3:       * priorityqueue         maxrss 1432.37 MB
        From worker 3:       * file                  maxrss 1438.73 MB
        From worker 3:       * read                  maxrss 1478.84 MB
        From worker 2:       * bitarray              maxrss 2526.68 MB
        From worker 2:       * version               maxrss 2526.68 MB
        From worker 2:       * resolve               maxrss 2526.68 MB
        From worker 2:       * pollfd                maxrss 2526.68 MB
        From worker 2:       * mpfr                  maxrss 2526.68 MB
        From worker 3:       * mmap                  maxrss 1478.84 MB
        From worker 2:       * broadcast             maxrss 2526.68 MB
        From worker 2:       * socket                maxrss 2526.68 MB
        From worker 2:       * floatapprox           maxrss 2526.68 MB
        From worker 3:       * complex               maxrss 1488.14 MB
        From worker 2:       * datafmt               maxrss 2526.68 MB
        From worker 2:       * regex                 maxrss 2526.68 MB
        From worker 2:       * float16               maxrss 2526.68 MB
        From worker 2:       * combinatorics         maxrss 2526.68 MB
        From worker 2:       * sysinfo               maxrss 2526.68 MB
        From worker 2:       * env                   maxrss 2526.68 MB
        From worker 2:       * rounding              maxrss 2526.68 MB
        From worker 3:       * reflection            maxrss 1496.32 MB
        From worker 3:       * mod2pi                maxrss 1496.77 MB
        From worker 3:       * euler                 maxrss 1505.23 MB
        From worker 3:       * show                  maxrss 1517.44 MB
        From worker 3:       * lineedit              maxrss 1517.64 MB
        From worker 3:       * replcompletions       maxrss 1517.82 MB
        From worker 3:       * repl                  maxrss 1518.42 MB
        From worker 3:       * replutil              maxrss 1518.79 MB
        From worker 3:       * sets                  maxrss 1522.95 MB
        From worker 2:       * ranges                maxrss 2593.79 MB
        From worker 2:       * goto                  maxrss 2689.30 MB
        From worker 2:       * llvmcall              maxrss 2691.29 MB
        From worker 2:       * grisu                 maxrss 2727.33 MB
        From worker 3:       * test                  maxrss 1526.15 MB
        From worker 3:       * meta                  maxrss 1526.19 MB
        From worker 3:       * stacktraces           maxrss 1526.29 MB
        From worker 3:       * profile               maxrss 1526.88 MB
        From worker 3:       * libgit2               maxrss 1561.61 MB
        From worker 3:       * docs                  maxrss 1562.80 MB
        From worker 3:       * markdown              maxrss 1562.80 MB
        From worker 2:       * nullable              maxrss 3024.51 MB
        From worker 3:       * base64                maxrss 1562.80 MB
        From worker 2:       * serialize             maxrss 3024.51 MB
Warning: threaded loop executed in order
        From worker 3:       * misc                  maxrss 1716.64 MB
        From worker 3:       * enums                 maxrss 1716.64 MB
        From worker 2:       * threads               maxrss 3024.51 MB
        From worker 2:       * i18n                  maxrss 3024.51 MB
        From worker 2:       * workspace             maxrss 3024.51 MB
        From worker 2:       * libdl                 maxrss 3024.51 MB
        From worker 2:       * int                   maxrss 3024.51 MB
        From worker 2:       * checked               maxrss 3024.51 MB
        From worker 2:       * intset                maxrss 3024.51 MB
        From worker 2:       * floatfuncs            maxrss 3024.51 MB
        From worker 3:       * cmdlineargs           maxrss 1716.64 MB
        From worker 3:       * inline                maxrss 1716.64 MB
        From worker 3:       * boundscheck           maxrss 1716.64 MB
        From worker 3:       * error                 maxrss 1716.64 MB
WARNING: Method definition ambig(Any, Integer) in module TestMain_ambiguous at /mypath/julia/dev/test/ambiguous.jl:7 overwritten at /mypath/julia/dev/test/ambiguous.jl:84.
        From worker 3:       * ambiguous            Skipping Base.<|
        From worker 3:  Skipping Base.>:
        From worker 3:  Skipping Base.active_repl
        From worker 3:  Skipping Base.active_repl_backend
        From worker 3:   maxrss 1716.64 MB
        From worker 3:       * cartesian             maxrss 1716.64 MB
        From worker 3:       * asmvariant            maxrss 1716.64 MB
        From worker 2:       * parallel              maxrss 3024.51 MB
        From worker 3:       * channels              maxrss 1800.05 MB
        From worker 3:       * dsp                   maxrss 1810.04 MB
        From worker 2:       * fft                   maxrss 3024.51 MB
        From worker 3:       * examples              maxrss 1810.04 MB
        From worker 1:       * compile               maxrss  213.62 MB

Test Summary: |     Pass  Broken     Total
  Overall     | 20025624      28  20025652
Test:                 | Total time (s): | GC time (s): | Percent in gc: | Allocated (MB): | RSS (MB):
linalg/qr             |   81.85         |    1.38      |    1.69        | 3156.39         |  350.59  
linalg/dense          |   36.01         |    0.55      |    1.52        |  876.45         |  379.46  
linalg/matmul         |   96.75         |    0.98      |    1.01        | 1970.51         |  445.50  
linalg/schur          |    7.70         |    0.11      |    1.48        |  201.03         |  450.61  
linalg/special        |   27.92         |    0.40      |    1.43        |  598.85         |  465.63  
linalg/triangular     |  263.64         |    8.68      |    3.29        | 8973.87         |  601.18  
linalg/eigen          |   21.60         |    0.22      |    1.00        |  402.02         |  474.65  
linalg/svd            |   10.05         |    0.18      |    1.79        |  279.56         |  480.48  
linalg/bunchkaufman   |   39.84         |    0.37      |    0.93        |  723.38         |  636.55  
linalg/lapack         |   26.28         |    0.66      |    2.50        |  737.81         |  491.08  
linalg/tridiag        |   25.46         |    0.48      |    1.90        |  648.18         |  675.62  
linalg/bidiag         |   33.48         |    0.36      |    1.09        |  446.46         |  494.70  
linalg/pinv           |    5.88         |    0.20      |    3.33        |  607.49         |  579.59  
linalg/givens         |    3.54         |    0.06      |    1.76        |   83.62         |  582.66  
linalg/diagonal       |   35.20         |    0.57      |    1.62        |  521.53         |  690.14  
linalg/cholesky       |   23.76         |    0.37      |    1.56        |  471.84         |  589.88  
linalg/symmetric      |    9.09         |    0.15      |    1.66        |  214.44         |  594.74  
linalg/generic        |    4.98         |    0.07      |    1.36        |   88.82         |  596.23  
linalg/lu             |   26.93         |    0.75      |    2.77        |  831.85         |  725.86  
linalg/uniformscaling |   38.41         |    0.46      |    1.20        |  308.75         |  599.59  
linalg/hessenberg     |    0.86         |    0.02      |    2.33        |   22.92         |  599.59  
linalg/lq             |   57.27         |    1.15      |    2.01        | 2312.36         |  805.55  
linalg/arnoldi        |   23.48         |    0.54      |    2.32        |  715.49         |  600.90  
sparse/sparse         |   97.91         |   40.41      |   41.28        | 1328.31         |  650.15  
sparse/sparsevector   |   99.81         |    1.55      |    1.55        | 1691.81         |  706.00  
sparse/umfpack        |    4.88         |    0.08      |    1.64        |   90.65         |  711.22  
sparse/cholmod        |   22.88         |    0.76      |    3.32        |  607.44         |  749.46  
sparse/spqr           |    2.11         |    0.06      |    2.87        |   68.79         |  749.46  
strings/basic         |    4.52         |    0.11      |    2.33        |  105.09         |  749.46  
strings/search        |    1.48         |    0.02      |    1.59        |   21.06         |  749.46  
strings/util          |    0.70         |    0.00      |    0.00        |    9.95         |  749.46  
strings/io            |    1.27         |    0.04      |    3.43        |   40.71         |  749.46  
strings/types         |    2.18         |    0.02      |    0.96        |   24.17         |  749.46  
unicode/UnicodeError  |    0.01         |    0.00      |    0.00        |    0.15         |  749.46  
unicode/utf8proc      |    1.88         |    0.04      |    2.31        |   36.69         |  749.46  
subarray              |  245.48         |    8.76      |    3.57        | 7427.48         | 1139.21  
unicode/utf8          |    2.37         |    1.09      |   46.06        |  851.04         |  795.05  
dates/adjusters       |    2.18         |    0.05      |    2.31        |   50.53         |  811.21  
dates/query           |    0.83         |    0.02      |    2.79        |   22.41         |  813.27  
dates/periods         |    9.51         |    0.19      |    1.95        |  169.42         |  813.80  
dates/accessors       |   13.66         |    8.11      |   59.38        | 2561.01         | 1797.75  
dates/rounding        |    0.70         |    0.00      |    0.00        |    9.65         | 1924.74  
dates/types           |    0.63         |    0.00      |    0.00        |    8.73         | 1934.20  
dates/io              |    2.35         |    0.00      |    0.00        |   46.45         | 1985.78  
dates/arithmetic      |    1.56         |    0.00      |    0.00        |   34.10         | 2020.57  
dates/conversions     |    0.85         |    0.00      |    0.00        |   10.86         | 2033.95  
dates/ranges          |   14.36         |    0.27      |    1.90        |  274.25         |  849.48  
inference             |    1.15         |    0.03      |    2.34        |   22.20         |  853.53  
keywordargs           |    1.42         |    0.03      |    1.76        |   19.28         |  853.63  
core                  |   77.72         |   44.60      |   57.38        | 1760.23         | 2526.68  
printf                |   13.66         |    0.26      |    1.91        |  200.52         | 2526.68  
char                  |    1.02         |    0.03      |    2.78        |   12.93         | 2526.68  
triplequote           |    0.02         |    0.00      |    0.00        |    0.21         | 2526.68
dict                  |   10.58         |    0.35      |    3.28        |  279.47         | 2526.68  
numbers               |   96.49         |    5.56      |    5.76        | 2580.89         | 1073.23  
iobuffer              |    0.70         |    0.00      |    0.00        |   10.71         | 1077.29  
hashing               |    5.06         |    0.15      |    2.93        |  105.74         | 2526.68  
staged                |    0.68         |    0.03      |    3.96        |   10.36         | 1079.45  
offsetarray           |   61.54         |    3.11      |    5.06        | 1056.44         | 2526.68  
tuple                 |    1.72         |    0.09      |    5.52        |   50.40         | 2526.68  
reduce                |    3.70         |    0.12      |    3.37        |   88.00         | 2526.68  
arrayops              |   70.59         |    2.51      |    3.55        | 1596.52         | 1292.36  
random                |   10.00         |    0.87      |    8.69        |  217.12         | 1292.36  
reducedim             |   14.38         |    0.47      |    3.28        |  326.85         | 2526.68  
intfuncs              |    0.99         |    0.00      |    0.00        |   14.88         | 2526.68  
simdloop              |    1.27         |    0.03      |    2.40        |   22.73         | 2526.68  
vecelement            |    1.81         |    0.06      |    3.54        |   24.54         | 2526.68  
blas                  |   29.25         |    0.52      |    1.77        |  412.39         | 2526.68  
abstractarray         |   75.92         |    4.28      |    5.64        | 2011.82         | 1292.36  
copy                  |    1.61         |    0.04      |    2.22        |   29.34         | 1292.36  
math                  |   65.49         |    3.61      |    5.51        | 1620.04         | 1356.03  
fastmath              |    3.27         |    0.06      |    1.94        |   43.97         | 1356.34  
functional            |    3.83         |    0.13      |    3.49        |   76.54         | 1356.71  
iterators             |   26.50         |    0.61      |    2.29        |  429.14         | 1375.49  
operators             |    2.07         |    0.13      |    6.33        |   41.53         | 1379.16  
path                  |    4.55         |    0.67      |   14.71        |   35.96         | 1379.16  
ccall                 |    5.07         |    0.14      |    2.82        |   75.99         | 1379.16  
parse                 |    1.61         |    0.00      |    0.00        |   18.21         | 1379.16  
loading               |    0.11         |    0.00      |    0.00        |    0.96         | 1379.16  
bigint                |    1.81         |    0.07      |    4.11        |   28.43         | 1379.16  
bigfloat              |    0.06         |    0.00      |    0.00        |    0.89         | 1379.16  
sorting               |   39.87         |    0.80      |    2.01        |  530.08         | 1395.66  
statistics            |    7.57         |    0.22      |    2.86        |  137.14         | 1403.16  
spawn                 |   20.18         |    0.21      |    1.03        |  114.79         | 1416.60  
backtrace             |    0.69         |    0.00      |    0.00        |    9.03         | 1421.23  
priorityqueue         |    1.41         |    0.14      |   10.20        |   90.87         | 1432.37  
file                  |   43.41         |    8.39      |   19.32        |   41.43         | 1438.73  
read                  |   14.98         |    0.43      |    2.84        |  342.42         | 1478.84  
bitarray              |  290.63         |    3.42      |    1.18        | 1736.96         | 2526.68  
version               |    1.22         |    0.00      |    0.00        |   15.37         | 2526.68  
resolve               |    4.15         |    0.24      |    5.79        |   82.77         | 2526.68  
pollfd                |    3.73         |    0.94      |   25.27        |   12.77         | 2526.68  
mpfr                  |    3.72         |    0.08      |    2.13        |   46.77         | 2526.68  
mmap                  |   29.86         |   26.93      |   90.21        |   50.63         | 1478.84  
broadcast             |   23.65         |    0.75      |    3.16        |  457.54         | 2526.68  
socket                |    3.05         |    0.07      |    2.38        |   39.74         | 2526.68  
floatapprox           |    0.27         |    0.00      |    0.00        |    3.95         | 2526.68  
complex               |   17.42         |    0.29      |    1.67        |  212.36         | 1488.14  
datafmt               |    4.62         |    0.11      |    2.36        |   78.86         | 2526.68  
regex                 |    0.49         |    0.04      |    7.40        |    7.57         | 2526.68  
float16               |    0.99         |    0.00      |    0.00        |   16.78         | 2526.68  
combinatorics         |    0.97         |    0.04      |    3.63        |   22.56         | 2526.68  
sysinfo               |    0.38         |    0.00      |    0.00        |    7.43         | 2526.68  
env                   |    0.54         |    0.00      |    0.00        |    7.27         | 2526.68  
rounding              |    0.77         |    0.04      |    4.68        |   13.21         | 2526.68  
reflection            |    5.93         |    0.18      |    2.99        |  112.34         | 1496.32  
mod2pi                |    0.13         |    0.00      |    0.00        |    2.21         | 1496.77  
euler                 |    0.59         |    0.03      |    5.77        |   20.43         | 1505.23  
show                  |   15.56         |    0.42      |    2.70        |  252.26         | 1517.44  
lineedit              |    3.57         |    0.11      |    3.02        |   75.90         | 1517.64  
replcompletions       |    3.48         |    0.11      |    3.28        |   53.84         | 1517.82  
repl                  |    5.55         |    0.11      |    2.04        |   71.16         | 1518.42  
replutil              |    4.87         |    0.11      |    2.30        |   59.66         | 1518.79
sets                  |    2.64         |    0.08      |    2.86        |   45.76         | 1522.95  
ranges                |   40.13         |   22.61      |   56.35        | 4875.82         | 2593.79  
goto                  |    0.07         |    0.00      |    0.00        |    0.26         | 2689.30  
llvmcall              |    0.15         |    0.00      |    0.00        |    1.51         | 2691.29  
grisu                 |    1.99         |    0.00      |    0.00        |   36.91         | 2727.33  
test                  |    7.56         |    0.10      |    1.38        |   58.36         | 1526.15  
meta                  |    0.31         |    0.00      |    0.00        |    3.45         | 1526.19  
stacktraces           |    0.97         |    0.00      |    0.00        |   10.89         | 1526.29  
profile               |    3.61         |    0.07      |    2.07        |   43.66         | 1526.88  
libgit2               |   15.02         |    0.25      |    1.64        |  128.64         | 1561.61  
docs                  |    3.90         |    0.91      |   23.25        |  109.42         | 1562.80  
markdown              |    3.74         |    0.13      |    3.37        |   60.09         | 1562.80  
nullable              |   29.53         |    1.52      |    5.15        |  318.01         | 3024.51  
base64                |    0.24         |    0.00      |    0.00        |    2.65         | 1562.80  
serialize             |    3.50         |    0.00      |    0.00        |   84.19         | 3024.51  
misc                  |   30.58         |    5.58      |   18.23        | 1307.88         | 1716.64  
enums                 |    1.25         |    0.00      |    0.00        |   19.50         | 1716.64  
threads               |   53.02         |    3.82      |    7.21        |   80.66         | 3024.51  
i18n                  |    0.01         |    0.00      |    0.00        |    0.07         | 3024.51  
workspace             |    0.90         |    0.00      |    0.00        |    0.55         | 3024.51  
libdl                 |    0.61         |    0.00      |    0.00        |    4.94         | 3024.51  
int                   |    3.09         |    0.10      |    3.15        |   30.77         | 3024.51  
checked               |   27.14         |    0.30      |    1.09        |  194.21         | 3024.51  
intset                |    0.60         |    0.04      |    6.27        |    7.68         | 3024.51  
floatfuncs            |    2.68         |    0.07      |    2.80        |   54.53         | 3024.51  
cmdlineargs           |  128.38         |    0.89      |    0.70        |   77.23         | 1716.64  
inline                |    0.28         |    0.00      |    0.00        |    1.29         | 1716.64  
boundscheck           |    2.77         |    0.00      |    0.00        |    1.13         | 1716.64  
error                 |    0.68         |    0.00      |    0.00        |    1.45         | 1716.64  
ambiguous             |    0.81         |    0.04      |    4.69        |   17.64         | 1716.64  
cartesian             |    0.00         |    0.00      |    0.00        |    0.04         | 1716.64  
asmvariant            |    0.01         |    0.00      |    0.00        |    0.06         | 1716.64  
parallel              |   77.31         |    0.04      |    0.05        |    3.60         | 3024.51  
channels              |   10.61         |    3.28      |   30.90        |  432.41         | 1800.05  
dsp                   |   10.31         |    1.30      |   12.64        |  287.00         | 1810.04  
fft                   |   21.49         |    2.70      |   12.57        |  686.26         | 3024.51  
examples              |   16.43         |    0.90      |    5.47        |  629.83         | 1810.04  
compile               |   16.98         |    0.02      |    0.14        |   53.75         |  213.62  
    SUCCESS
tkelman commented 7 years ago

Don't see how your failure could be compiler version specific. It looks like the patches are not being tracked correctly for building clang.

lascott commented 7 years ago

@tkelman could you expand your thoughts for me? I don't yet know enough to pick up quickly.

You are correct that this is driven by clang.jl as the following commentary there indicates the need for building llvm into julia to get it to work. wrap_c.init fails

Once I go down the rabbit hole of BUILD_LLVM_CLANG=1 in Make.user in Ubuntu 14.04 LTS, Mint 17.3 xfce, I get into a few problems that go away for my simple Make.user with gcc-5

An excerpt from the above mentioned issue in clang.jl:

... Building Clang as part of Julia (with BUILD_LLVMCLANG=1 in Make.user) works fine. On Wed, Dec 2, 2015 at 10:23 AM, Kaj Wiik notifications@github.com wrote: Do I understand correctly that in order to use Clang, one must build Julia always from source? ihnorton commented on Dec 2, 2015 Yes, unless you know some way to get libclang compiled against a different LLVM version to load in the same process (I tried various RTLD options and didn't have any luck).

Curious to know if others using 14.04 have had problems with BUILD_LLVM_CLANG.

Both Clang.jl:OSX: Let Homebrew install llvm automatically and Clang.jl build 166 deal with problems in getting it to work in travis. I suspect I will move to Clang.jl for further ideas, but getting julia to build with BUILD_LLVM_CLANG belongs here.

tkelman commented 7 years ago

the memory issue may behave differently for different compiler versions, but "can't find file to patch" shouldn't

lascott commented 7 years ago

Thank you. I will close this as it is now working under gcc5. However, I will summarise what I found first, then close.