clang doesnt use library search path that it was built with


Bugzilla Link	20510
Version	trunk
OS	Linux
Attachments	config log
Reporter	LLVM Bugzilla Contributor
CC	@DougGregor,@zygoloid,@rnk,@viccpp

Extended Description

The symptom of this problem is that clang builds can fail with an error similar to:

clang++: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by clang++)

This appears to be because clang does not "remember" the search path it was built with. In this case, clang was built w/gcc 4.9.0, which is installed in /usr/local.

/shared/clang $ /usr/local/bin/gcc -print-search-dirs
install: /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/
programs: =/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/
libraries: =/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/../lib64/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../lib64/:/lib/x86_64-unknown-linux-gnu/4.9.0/:/lib/../lib64/:/usr/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/../lib64/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../:/lib/:/usr/lib/

/shared/clang $ Release+Asserts/bin/clang++ -print-search-dirs
programs: =/shared/clang/Release+Asserts/bin:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/bin
libraries: =/shared/clang/Release+Asserts/bin/../lib/clang/3.6.0:/usr/lib/gcc/x86_64-redhat-linux/4.4.7:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64:/lib/../lib64:/usr/lib/../lib64:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../..:/shared/clang/Release+Asserts/bin/../lib:/lib:/usr/lib

As you can see, clang is picking up the library search paths from the system default compiler (gcc 4.4), rather than the search paths with which clang was built (gcc 4.9).

Using LD_LIBRARY_PATH is not an option -- our build scripts deliberately unset LD_LIBRARY_PATH before linking to ensure that only specifically named library paths are searched.

I'm attaching the config.log, which appears to show clang picking up gcc 4.9.0 for the build (as expected).

The command used to build clang is:

make clean distclean;CFLAGS="-std=gnu89" ../llvm/configure --prefix=/build/share/clang --enable-optimized --enable-targets=x86_64 && make

At this point I'm not necessarily looking for a fix, but an indication whether this is a legitimate problem with clang, or possibly a problem on our end.

Thanks!

OK, I think I've finally figured this out.

I wrote up my findings in this blog post: http://btorpey.github.io/blog/2015/01/02/building-clang/

I'd appreciate it if Reid et al. could take a look and offer any suggestions, corrections, etc.

Thanks!

I think it would be good to figure out why the --with-gcc-toolchain= option isn't working. As far as I can tell, it does exactly what you want: it bakes in the path to a gcc installation so that it can find it for headers and libraries. This would remove the need to thread --gcc-toolchain flags through all your build systems, which seems like a win.

Unfortunately, the "baking" is done with rpath. The policy in our shop is that NOTHING goes to QA or prod with an rpath (or runpath) in it. (I happen to agree, but even if I didn't that wouldn't matter, it's not going to change).

So, we know we need to have compiler support libraries on LD_LIBRARY_PATH, and that's OK -- the deployment tools take care of that. What's not so great is having to add compiler support libraries for the compiler that was used to compile the code that we're deploying. It would be one thing if we could get that information from clang somehow, but when we query clang for search dirs, it responds with the *system compiler's search dirs. On RH6, of course, the system compiler can't even compile clang, so that does no good at all.

Are you still having issues with programs linked by clang not finding the more recent version of libstdc++? In that case, I don't think clang is behaving differently from gcc in this respect. Any binaries produced by gcc from a non-standard installation path will link against libraries that the loader won't find by default.

Yes, and as noted above we're OK with that. It's the second level of indirection (which compiler compiled the the compiler that compiled the code itself) that is a major hassle.

There are three ways that glibc's loader finds libraries needed by a program:

Looks in standard install paths (/usr/lib, /usr/lib64, etc)

Looks in LD_LIBRARY_PATH

Looks in RPATH in binary

I think the order is reversed, but no matter in our case, since co. policy forbids using RPATH in QA or prod.

You need to pick one of those approaches to help any program find dynamic libraries, but it seems like you've rejected all of them.

I'm open to anything at this point, including using libcxx to eliminate depencencies on gcc entirely (assuming that it will).

If necessary we can hard- code the dependency between a particular version of clang and the gcc version that was used to build it, and add BOTH sets of support libraries to LD_LIBRARY_PATH at runtime, but that seems like a hack, and error-prone -- which is why I'm trying to find another approach.

How do other programs in your environment find the dynamic libraries they need? Consider the program /build/share/bin/foo which links against /build/share/lib64/libbar.so.

In that case we add /build/share/lib64 to LD_LIBRARY_PATH at runtime. But again, that is NOT done using rpath, and the environment settings (including LD_LIBRARY_PATH) are controlled by an operations group separate from the dev group, which gives at least a modicum of security and traceability to the process.

I think it would be a good idea to raise these kinds of broad usage questions on cfe-dev.

I will give that a shot -- thanks for the suggestion! Although my guess is that anything that comes out of that will take a while to percolate.

Our bug tracker is configured to only send updates to llvmbugs@ on issue status changes, so further replies to this bug only go to people on the cc list (me), and I'm out of ideas. :(

Well, hopefully the above gives you a better understanding of the pickle we're in.

Going back to square one, the issue was that clang generates code that requires library versions that are not possible to discover by querying clang itself (e.g., by -print-search-dirs). If we could at least get that information, we could use it to generate some kind of manifest that could be used to ensure that dependent libraries are installed and configured at runtime.

As it stands, the values returned by -print-search-dirs are incorrect/useless in the case where clang was built with a compiler other than the standard system compiler.

In my case, I build clang with /build/share/gcc/4.9.0/bin/g++ -- but what I get from clang is:

$ lsb_release -a
LSB Version:    :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description:    CentOS release 6.5 (Final)
Release:    6.5
Codename:   Final
$ which gcc
/build/share/gcc/4.9.0/bin/gcc
$ which clang
/build/share/clang/3.6.0-svn/bin/clang
$ clang -print-search-dirs
programs: =/build/share/clang/3.6.0-svn/bin:/shared/build/share/clang/3.6.0-svn/bin:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/bin
libraries: =/shared/build/share/clang/3.6.0-svn/bin/../lib/clang/3.6.0:/usr/lib/gcc/x86_64-redhat-linux/4.4.7:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64:/lib/../lib64:/usr/lib/../lib64:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../..:/shared/build/share/clang/3.6.0-svn/bin/../lib:/lib:/usr/lib

However, clang DOES seem to have this tucked away somewhere:

$ strings `which clang` | grep "/build/share/gcc"
/build/share/gcc/4.9.0/lib64:$ORIGIN/../lib

AND, it shows up as a dependency (although for the life of me I can't get either readelf or objdump to show it to me :-(

$ unset LD_LIBRARY_PATH
$ ldd `which clang`
    linux-vdso.so.1 =>  (0x00007fff13f47000)
    librt.so.1 => /lib64/librt.so.1 (0x0000003dae000000)
    libdl.so.2 => /lib64/libdl.so.2 (0x0000003dad400000)
    libtinfo.so.5 => /lib64/libtinfo.so.5 (0x0000003dbfc00000)
    libatomic.so.1 => /build/share/gcc/4.9.0/lib64/libatomic.so.1 (0x00007f5233b36000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003dad800000)
    libz.so.1 => /lib64/libz.so.1 (0x0000003dadc00000)
    libstdc++.so.6 => /build/share/gcc/4.9.0/lib64/libstdc++.so.6 (0x00007f5233823000)
    libm.so.6 => /lib64/libm.so.6 (0x0000003dacc00000)
    libgcc_s.so.1 => /build/share/gcc/4.9.0/lib64/libgcc_s.so.1 (0x00007f523360c000)
    libc.so.6 => /lib64/libc.so.6 (0x0000003dad000000)
    /lib64/ld-linux-x86-64.so.2 (0x0000003dac800000)

Anyway, thanks for taking the time to read this, as well as for all the great clang stuff that we can't wait to deploy!

I think it would be good to figure out why the --with-gcc-toolchain= option isn't working. As far as I can tell, it does exactly what you want: it bakes in the path to a gcc installation so that it can find it for headers and libraries. This would remove the need to thread --gcc-toolchain flags through all your build systems, which seems like a win.

Are you still having issues with programs linked by clang not finding the more recent version of libstdc++? In that case, I don't think clang is behaving differently from gcc in this respect. Any binaries produced by gcc from a non-standard installation path will link against libraries that the loader won't find by default.

There are three ways that glibc's loader finds libraries needed by a program:

Looks in standard install paths (/usr/lib, /usr/lib64, etc)
Looks in LD_LIBRARY_PATH
Looks in RPATH in binary

You need to pick one of those approaches to help any program find dynamic libraries, but it seems like you've rejected all of them. How do other programs in your environment find the dynamic libraries they need? Consider the program /build/share/bin/foo which links against /build/share/lib64/libbar.so.

I think it would be a good idea to raise these kinds of broad usage questions on cfe-dev. Our bug tracker is configured to only send updates to llvmbugs@ on issue status changes, so further replies to this bug only go to people on the cc list (me), and I'm out of ideas. :(

Up till now, I've been working more or less solo w/clang, so the fact that I need to jump through hoops to get it working isn't too bad. However, now that we're getting ready to roll out clang support to our dev & QA environments, it's become a real pain.

Given our insistence on stripping rpath from executables, I can only think of two worakable approaches to dealing with the fact that clang itself depends on gcc libraries, and that clang generates code that also depends on gcc libraries.

The clang install script could copy required libraries at install time from the source directories to the install directories. I don't know if this would run afoul of licensing terms for gcc, but if not it could be an attractive alternative for people who are looking for a self-contained clang distribution. We already need to know where to get library support for the compiler used, and our configuration management can properly set LD_LIBRARY_PATH at runtime to the clang directories.
Switch to using llvm's libcxx and libcxxabi support, and eliminate all dependencies on gcc. This appears to be easier said than done -- building with the most recent version (222451) gives compile errors. So, before going down that rabbit hole, I have a couple of questions:

Is clang's support for libcxx and libcxxabi robust enough to make this a viable approach? (Note that we're on RH/CentOS 6.5).
Would moving to libcxx and libcxxxabi eliminate ALL dependencies on gcc libraries for code generated with clang? Otherwise, there's obviously no point.
What about third-party libraries? Can code built w/clang+libcxx/libcxxabi interoperate with code that was built w/gcc/libstdc++? Are there any restrictions -- e.g., can code built with one platform pass objects like strings to code built with the other platform? (For instance, this has been a problem for us in the past when building with _GLIBCXX_DEBUG).

I'm not necessarily looking for a fix here (although that would be lovely), but I would certainly appreciate some guidance on what is likely to be the most effective approach.

Thanks again!

To follow up on earlier post, I've followed all the instructions and have working installs (sort of).

To make things work, I need to do the following:

add --gcc-toolchain=... to compile and link flags when building my project;

In this case, the value specified for gcc-toolchain is hard-coded in makefile (actualy CMakeLists.txt as we're starting to use CMake for new stuff).

It would be nice if we could at least get the correct gcc-toolchain value from clang in such a way that we could parse it at build-time rather than having to hard-code it in make file. Unfortunately, both -print-search-dirs and -print-libgcc-file-name return the system compiler (in this case gcc 4.4.7), even though clang was built with gcc 4.9.0.

(FWIW, gcc is installed in a non-standard location as well -- our convention is to have all build tools and 3rd-party dependencies in /build/share//. ).

Add the lib64 directory from the gcc-toolchain to the LD_LIBRARY_PATH for any executables built with that version of clang

Otherwise, we end up with /usr/lib64/libstdc++.so.6: versionGLIBCXX_3.4.15' not found (required by ...)` errors for any executables that were built with that version of clang.

Before you go there, we deliberately strip rpath from executables for a couple of reasons:

We want to ensure for any code that is running in production that we know EXACTLY where the dependencies come from. For that reason we have very tight controls on what can and cannot be in the environment for a running process, and we do that by controlling what is in PATH and LD_LIBRARY_PATH on a per-component basis.
The directory structure used in QA & production is very different from the directory structure used in dev -- partly for reasons that may not make sense, but also for the reason mentioned above: it just makes it harder for an implicit dependency to "sneak" into the environment.

So, maybe I'm doing things wrong here, but based on reading the docs, and from what you've said earlier I don't think I am.

And if I'm not doing things wrong, this seems like an awful lot of hoops to jump through that are specific to clang.

FWIW, we've been doing things this way for about 7-8 years now using gcc (non-standard directory structure, locked-down PATH/LD_LIBRARY_PATH, no RPATH, etc.) and these have not been issues.

If there's a better way to do things (other than changing around our procedures, which frankly are not going to change any time soon), I'd love to know what it is.

Again, thanks for your help, suggestions and patience! This is not a knock on clang (which keeps getting better every time I take another look -- I can't wait to start working with the static thread safety analysis, for instance), but this whole experience has been a little frustrating.

OK, back from vacation, co. being sold, etc.

A couple of points:

Adding linker flags to force clang to find the right libstdc++ doesnt appear to work

Linking CXX executable mdl
cd /home/btorpey/work/bustalkDev/nsb/api/mdl && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/mdl.dir/link.txt --verbose=1
/home/btorpey/work/bustalkDev/admin/../dev/link.sh /usr/local/bin/clang++   -L/usr/local/bin/../lib64  -Dartf50401  -ggdb -fno-omit-frame-pointer -fPIC -D_REENTRANT   -Wall -Wextra -Wformat -Wformat-nonliteral -Wno-reorder -Wno-c++11-extensions -Wno-documentation-unknown-command -Wno-gnu -Wno-unused-parameter -Wno-padded -D_DEBUG -O0 -DJTI_NO_STD_EXCEPTION    CMakeFiles/mdl.dir/msggen.cpp.o CMakeFiles/mdl.dir/mdl-parse.cpp.o CMakeFiles/mdl.dir/mdl-lex.cpp.o CMakeFiles/mdl.dir/int16Type.cpp.o CMakeFiles/mdl.dir/int32Type.cpp.o CMakeFiles/mdl.dir/int64Type.cpp.o CMakeFiles/mdl.dir/byteType.cpp.o CMakeFiles/mdl.dir/stringType.cpp.o CMakeFiles/mdl.dir/fixType.cpp.o CMakeFiles/mdl.dir/longStringType.cpp.o CMakeFiles/mdl.dir/pointerType.cpp.o CMakeFiles/mdl.dir/externType.cpp.o CMakeFiles/mdl.dir/boolType.cpp.o CMakeFiles/mdl.dir/uint64Type.cpp.o  -o mdl -L/usr/local/bin/../lib64 -rdynamic -L/usr/local/lib /usr/local/lib/libboost_thread.so -lboost_program_options -lboost_system 
/usr/local/bin/clang++: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by /usr/local/bin/clang++)
/usr/local/bin/clang++: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by /usr/local/bin/clang++)
/usr/local/bin/clang++: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /usr/local/bin/clang++)
/usr/local/bin/clang++: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /usr/local/bin/clang++)
make[2]: *** [nsb/api/mdl/mdl] Error 1
make[2]: Leaving directory `/shared/btorpey/work/bustalkDev'
make[1]: *** [nsb/api/mdl/CMakeFiles/mdl.dir/all] Error 2
make[1]: Leaving directory `/shared/btorpey/work/bustalkDev'
make: *** [all] Error 2

As you can see, the correct -L parameter is being passed to clang++, but it would appear that this is ignored in the case of libstdc++, and that clang uses a burned-in path instead.

What I ended up doing is taking your suggestion to build clang w/rpath, as documented here: http://llvm.org/docs/GettingStarted.html#getting-a-modern-host-c-toolchain

FWIW, the build command looks like this (for building clang using gcc 4.9.0, with gcc installed in default location of /usr/local):

CC=/usr/local/gcc CXX=/usr/local/g++ cmake ../llvm -DCMAKE_CXX_LINK_FLAGS="-Wl,-rpath,/usr/local/lib64 -L/usr/local/lib64" -DCMAKE_BUILD_TYPE="Release" -DLLVM_TARGETS_TO_BUILD="X86"

My original point is still valid, though, I believe:

I shouldn't have to tell clang where to get the libraries that he's going to use to generate code, since he already knows that.
But, if I don't tell him, he doesn't work. (To be precise, clang generates code that clang itself doesn't know how to link).

I'm just saying ....

P.S. I'm going to be on vacation for the next week or so, so if you reply and dont hear back from me right away, that's the reason.

Thanks again for all your help!

My point is that I think clang should behave more like gcc in this regard. If you look at the following info for gcc 4.9.0, you can see that even though it was built with the system compiler, it has the "correct" search paths baked in -- i.e., the paths corresponding to the code that itself will generate.

/home/btorpey $ which gcc
/usr/local/bin/gcc
/home/btorpey $ gcc --version
gcc (GCC) 4.9.0
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

/home/btorpey $ gcc -print-search-dirs
install: /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/
programs: =/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/
libraries: =/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/../lib64/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../lib64/:/lib/x86_64-unknown-linux-gnu/4.9.0/:/lib/../lib64/:/usr/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/../lib64/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../:/lib/:/usr/lib/
/home/btorpey $

OTOH, clang has the system compiler search paths baked in, even though clang is generating code that does not reference those libraries -- instead clang is generating code that depends on the libraries of the compiler with which it was built.

/home/btorpey $ which clang
/shared/clangwgcc490/Release+Asserts/bin/clang
/home/btorpey $ clang --version
clang version 3.6.0 (trunk 214514) (llvm/trunk 214513)
Target: x86_64-unknown-linux-gnu
Thread model: posix
/home/btorpey $ clang -print-search-dirs
programs: =/shared/clangwgcc490/Release+Asserts/bin:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/bin
libraries: =/shared/clangwgcc490/Release+Asserts/bin/../lib/clang/3.6.0:/usr/lib/gcc/x86_64-redhat-linux/4.4.7:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64:/lib/../lib64:/usr/lib/../lib64:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../..:/shared/clangwgcc490/Release+Asserts/bin/../lib:/lib:/usr/lib
/home/btorpey $

So, as to your statement "Clang is trying to act in exactly the same way as the "system" compiler", I submit that that is the wrong goal -- clang should just act "like itself" and generate code that is internally consistent with the libraries used by the compiler.

I understand that we can pass the required libraries to the linker, but that is a problem for a couple of reasons:

It's a maintenance nightmare -- we use a lot of different compilers and OS's, so this is just one more thing that needs to change.
How do we know what search paths to add? That doesn't appear to be documented anywhere, and -print-search-dirs is not going to tell us ;-)

Last but not least, I tried rebuilding with the -with-gcc-toolchain optiion, but no joy there (let me know if you want the complete output), but that appears to crash gcc:

]0;[/shared/clang] /shared/clang $ export PATH=/usr/local/bin:$PATH
]0;[/shared/clang] /shared/clang $ gcc --version
gcc (GCC) 4.9.0
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

]0;[/shared/clang] /shared/clang $ make clean dist-clean;CFLAGS="-std=gnu89" ../llvm/configure --prefix=/build/share/clang --enable-optimized --enable-targets=x86_64 --with-gcc-toolchain=/usr/local && make
make: *** No rule to make target `clean'.  Stop.
checking for clang... no
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for clang++... no
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C preprocessor... gcc -E
checking whether GCC or Clang is our host compiler... gcc
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking type of operating system we're going to host on... Linux
checking type of operating system we're going to target... Linux
checking target architecture... x86_64
checking whether GCC is new enough... yes
checking optimization flags... -O3
checking for GNU make... make
checking whether ln -s works... yes
checking for nm... /usr/bin/nm
checking for cmp... /usr/bin/cmp
checking for cp... /bin/cp
checking for date... /bin/date
checking for find... /usr/bin/find
checking for grep... /bin/grep
checking for mkdir... /bin/mkdir
checking for mv... /bin/mv
checking for ranlib... ranlib
checking for ar... ar
checking for rm... /bin/rm
checking for sed... /bin/sed
checking for tar... /bin/tar
checking for pwd... /bin/pwd
checking for dot... /usr/bin/dot
checking for a BSD-compatible install... /usr/bin/install -c
checking for bzip2... /usr/bin/bzip2
checking for cat... /bin/cat
checking for doxygen... /usr/bin/doxygen
checking for groff... /usr/bin/groff
checking for gzip... /usr/bin/gzip
checking for pdfroff... no
checking for zip... /usr/bin/zip
checking for ocamlc... no
checking for ocamlopt... no
checking for ocamldep... no
checking for ocamldoc... no
checking for gas... no
checking for as... /usr/bin/as
checking for linker version... 2.20.51.0.2
checking for compiler -Wl,-R<path> option... yes
checking for compiler -rdynamic option... yes
checking for compiler -Wl,--version-script option... yes
checking for grep that handles long lines and -e... (cached) /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking errno.h usability... yes
checking errno.h presence... yes
checking for errno.h... yes
checking tool compatibility... ok
checking optional compiler flags... -Wno-variadic-macros -Wno-missing-field-initializers   -Wno-maybe-uninitialized
checking for python... /usr/bin/python
checking for python >= 2.5... /usr/bin/python (2.6.6)
checking for sin in -lm... yes
checking for library containing dlopen... -ldl
checking for library containing clock_gettime... -lrt
checking for library containing setupterm... -ltinfo
checking for library containing el_init... no
checking for library containing mallinfo... none required
checking for pthread_mutex_init in -lpthread... yes
checking for library containing pthread_mutex_lock... none required
checking for library containing pthread_rwlock_init... none required
checking for library containing pthread_getspecific... none required
checking for compress2 in -lz... yes
checking for xml2-config... xml2-config
checking for libxml2 includes... -I/usr/include/libxml2
checking for xmlReadFile in -lxml2... yes
checking for dirent.h that defines DIR... yes
checking for library containing opendir... none required
checking for MAP_ANONYMOUS vs. MAP_ANON... yes
checking whether stat file-mode macros are broken... no
checking for sys/wait.h that is POSIX.1 compatible... yes
checking whether time.h and sys/time.h may both be included... yes
checking how to run the C++ preprocessor... g++ -E
checking cxxabi.h usability... yes
checking cxxabi.h presence... yes
checking for cxxabi.h... yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking execinfo.h usability... yes
checking execinfo.h presence... yes
checking for execinfo.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking for inttypes.h... (cached) yes
checking link.h usability... yes
checking link.h presence... yes
checking for link.h... yes
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking setjmp.h usability... yes
checking setjmp.h presence... yes
checking for setjmp.h... yes
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking for stdint.h... (cached) yes
checking termios.h usability... yes
checking termios.h presence... yes
checking for termios.h... yes
checking for unistd.h... (cached) yes
checking utime.h usability... yes
checking utime.h presence... yes
checking for utime.h... yes
checking sys/mman.h usability... yes
checking sys/mman.h presence... yes
checking for sys/mman.h... yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking sys/resource.h usability... yes
checking sys/resource.h presence... yes
checking for sys/resource.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking sys/uio.h usability... yes
checking sys/uio.h presence... yes
checking for sys/uio.h... yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking malloc/malloc.h usability... no
checking malloc/malloc.h presence... no
checking for malloc/malloc.h... no
checking mach/mach.h usability... no
checking mach/mach.h presence... no
checking for mach/mach.h... no
checking valgrind/valgrind.h usability... yes
checking valgrind/valgrind.h presence... yes
checking for valgrind/valgrind.h... yes
checking fenv.h usability... yes
checking fenv.h presence... yes
checking for fenv.h... yes
checking whether FE_ALL_EXCEPT is declared... yes
checking whether FE_INEXACT is declared... yes
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking CrashReporterClient.h usability... no
checking CrashReporterClient.h presence... no
checking for CrashReporterClient.h... no
checking __crashreporter_info__... no
checking for HUGE_VAL sanity... yes
checking for pid_t... yes
checking for size_t... yes
checking whether struct tm is in sys/time.h or time.h... time.h
checking for int64_t... yes
checking for uint64_t... yes
checking for backtrace... yes
checking for ceilf... yes
checking for floorf... yes
checking for roundf... yes
checking for rintf... yes
checking for nearbyintf... yes
checking for getcwd... yes
checking for powf... yes
checking for fmodf... yes
checking for strtof... yes
checking for round... yes
checking for log... yes
checking for log2... yes
checking for log10... yes
checking for exp... yes
checking for exp2... yes
checking for getpagesize... yes
checking for getrusage... yes
checking for getrlimit... yes
checking for setrlimit... yes
checking for gettimeofday... yes
checking for isatty... yes
checking for mkdtemp... yes
checking for mkstemp... yes
checking for mktemp... yes
checking for posix_spawn... yes
checking for pread... yes
checking for realpath... yes
checking for sbrk... yes
checking for setrlimit... (cached) yes
checking for strerror... yes
checking for strerror_r... yes
checking for setenv... yes
checking for strtoll... yes
checking for strtoq... yes
checking for sysconf... yes
checking for malloc_zone_statistics... no
checking for setjmp... yes
checking for longjmp... yes
checking for sigsetjmp... no
checking for siglongjmp... yes
checking for writev... yes
checking for futimes... yes
checking for futimens... yes
checking if printf has the %a format character... yes
checking for srand48/lrand48/drand48 in <stdlib.h>... yes
checking whether arc4random is declared... no
checking whether strerror_s is declared... no
checking for isnan in <math.h>... yes
checking for isnan in <cmath>... yes
checking for std::isnan in <cmath>... yes
checking for isinf in <math.h>... yes
checking for isinf in <cmath>... yes
checking for std::isinf in <cmath>... yes
checking for finite in <ieeefp.h>... no
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for getpagesize... (cached) yes
checking for working mmap... yes
checking for mmap of files... yes
checking if /dev/zero is needed for mmap... no
checking for GCC atomic builtins... yes
checking for 32-bit userspace on 64-bit system... no
checking for __dso_handle... yes
checking for compiler -fvisibility-inlines-hidden option... yes
configure: creating ./config.status
config.status: creating include/llvm/Config/Targets.def
config.status: creating include/llvm/Config/AsmPrinters.def
config.status: creating include/llvm/Config/AsmParsers.def
config.status: creating include/llvm/Config/Disassemblers.def
config.status: creating Makefile.config
config.status: creating llvm.spec
config.status: creating docs/doxygen.cfg
config.status: creating tools/clang/docs/doxygen.cfg
config.status: creating bindings/ocaml/llvm/META.llvm
config.status: creating include/llvm/Config/config.h
config.status: creating include/llvm/Config/llvm-config.h
config.status: creating include/llvm/Support/DataTypes.h
config.status: creating tools/clang/include/clang/Config/config.h
config.status: executing setup commands
config.status: executing Makefile commands
config.status: executing Makefile.common commands
config.status: executing examples/Makefile commands
config.status: executing lib/Makefile commands
config.status: executing test/Makefile commands
config.status: executing test/Makefile.tests commands
config.status: executing unittests/Makefile commands
config.status: executing tools/Makefile commands
config.status: executing utils/Makefile commands
config.status: executing projects/Makefile commands
config.status: executing bindings/Makefile commands
config.status: executing bindings/ocaml/Makefile.ocaml commands
llvm[0]: Constructing LLVMBuild project information.
...
llvm[4]: Compiling RecursiveASTVisitorTest.cpp for Release+Asserts build
g++: internal compiler error: Killed (program cc1plus)
0x409fd4 execute
    ../../gcc/gcc.c:2848
0x40a384 do_spec_1
    ../../gcc/gcc.c:4652
0x40cc46 process_brace_body
    ../../gcc/gcc.c:5935
0x40cc46 handle_braces
    ../../gcc/gcc.c:5849
0x40b1f9 do_spec_1
    ../../gcc/gcc.c:5306
0x40cc46 process_brace_body
    ../../gcc/gcc.c:5935
0x40cc46 handle_braces
    ../../gcc/gcc.c:5849
0x40b1f9 do_spec_1
    ../../gcc/gcc.c:5306
0x40af63 do_spec_1
    ../../gcc/gcc.c:5421
0x40cc46 process_brace_body
    ../../gcc/gcc.c:5935
0x40cc46 handle_braces
    ../../gcc/gcc.c:5849
0x40b1f9 do_spec_1
    ../../gcc/gcc.c:5306
0x40cc46 process_brace_body
    ../../gcc/gcc.c:5935
0x40cc46 handle_braces
    ../../gcc/gcc.c:5849
0x40b1f9 do_spec_1
    ../../gcc/gcc.c:5306
0x40cc46 process_brace_body
    ../../gcc/gcc.c:5935
0x40cc46 handle_braces
    ../../gcc/gcc.c:5849
0x40b1f9 do_spec_1
    ../../gcc/gcc.c:5306
0x40cc46 process_brace_body
    ../../gcc/gcc.c:5935
0x40cc46 handle_braces
    ../../gcc/gcc.c:5849
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
/bin/rm: cannot remove `/shared/clang/tools/clang/unittests/Tooling/Release+Asserts/RecursiveASTVisitorTest.d.tmp': No such file or directory
make[4]: *** [/shared/clang/tools/clang/unittests/Tooling/Release+Asserts/RecursiveASTVisitorTest.o] Error 1
make[4]: Leaving directory `/shared/clang/tools/clang/unittests/Tooling'
make[3]: *** [Tooling/.makeall] Error 2
make[3]: Leaving directory `/shared/clang/tools/clang/unittests'
make[2]: *** [all] Error 1
make[2]: Leaving directory `/shared/clang/tools/clang'
make[1]: *** [clang/.makeall] Error 2
make[1]: Leaving directory `/shared/clang/tools'
make: *** [all] Error 1

Reid:

Dont get me wrong -- I’m nuts about clang in general, and especially the static analysis, sanitizers and related bits that help people, including us, build software that is more reliable and correct. And I’m very grateful to those, like you, who make that possible.

Thanks!

With all due respect, however, what you suggest is a workaround, not a fix, and not a particularly good one either -- it requires manually adding the library search paths, not of the compiler, but of the compiler that the compiler was built with. Manually adding library search paths would be borked to begin with, but having to add the libraries that the compiler was built with is completely ridiculous.

I don't think this is too bad, it's just one more linker flag on top of installing and using a new version of GCC, which is already a tall order. Setting an rpath is the standard way of using any library that is not installed into /usr/lib. Clang is just a normal application that wants to use a newer copy of libstdc++.

Please reopen this so that this bug (and it is a bug) is visible to someone who wants to work on fixes for the compiler.

What action needs to be taken to make this easy? This really has more to do with what gcc does when you compile a C++ app with it. It will add -L flags to the link line, but it will not set an rpath.

In the meantime, I’m rebuilding clang with the native compiler (gcc 4.4.7) and hope that will work. I’ll post back here what I find out.

OK, clang wont build w/gcc 4.4.7 -- not surprising, as that's well documented.

OTOH, that does make it more egregious that clang insists on using that compiler as its default search path.

Clang is trying to act in exactly the same way as the "system" compiler, which it assumes is installed in /usr, and will search /usr/include, /usr/lib, and a variety of paths off of there. If you have a more modern gcc and libstdc++ toolchain that you want it to use, I believe you can set GCC_INSTALL_PREFIX at configure time. I don't know how to do that in autoconf, though. My best guess is --with-gcc-toolchain.

OK, clang wont build w/gcc 4.4.7 -- not surprising, as that's well documented.

OTOH, that does make it more egregious that clang insists on using that compiler as its default search path.

Reid:

Dont get me wrong -- I’m nuts about clang in general, and especially the static analysis, sanitizers and related bits that help people, including us, build software that is more reliable and correct. And I’m very grateful to those, like you, who make that possible.

With all due respect, however, what you suggest is a workaround, not a fix, and not a particularly good one either -- it requires manually adding the library search paths, not of the compiler, but of the compiler that the compiler was built with. Manually adding library search paths would be borked to begin with, but having to add the libraries that the compiler was built with is completely ridiculous.

Please reopen this so that this bug (and it is a bug) is visible to someone who wants to work on fixes for the compiler.

In the meantime, I’m rebuilding clang with the native compiler (gcc 4.4.7) and hope that will work. I’ll post back here what I find out.

Best Regards,

Bill Torpey

I tried to document this harder in r214565.

Set the rpath instead. Add -Wl,-rpath,/usr/local/lib64 or something like it to your link flags.

This is documented in http://llvm.org/docs/GettingStarted.html#getting-a-modern-host-c-toolchain, but it could probably be improved.

@llvm/issue-subscribers-clang-driver

Author: None (llvmbot)

| | | | --- | --- | | Bugzilla Link | [20510](https://llvm.org/bz20510) | | Version | trunk | | OS | Linux | | Attachments | [config log](https://user-images.githubusercontent.com/60944935/143749817-ad91c973-2f46-429a-bf28-9e0def3a6afb.gz) | | Reporter | LLVM Bugzilla Contributor | | CC | @DougGregor,@zygoloid,@rnk,@viccpp | ## Extended Description The symptom of this problem is that clang builds can fail with an error similar to: ``` clang++: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by clang++) ``` This appears to be because clang does not "remember" the search path it was built with. In this case, clang was built w/gcc 4.9.0, which is installed in `/usr/local`. ```console /shared/clang $ /usr/local/bin/gcc -print-search-dirs install: /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/ programs: =/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/libexec/gcc/x86_64-unknown-linux-gnu/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/ libraries: =/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/../lib64/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../x86_64-unknown-linux-gnu/4.9.0/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../lib64/:/lib/x86_64-unknown-linux-gnu/4.9.0/:/lib/../lib64/:/usr/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/../lib64/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/:/usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../:/lib/:/usr/lib/ /shared/clang $ Release+Asserts/bin/clang++ -print-search-dirs programs: =/shared/clang/Release+Asserts/bin:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/bin libraries: =/shared/clang/Release+Asserts/bin/../lib/clang/3.6.0:/usr/lib/gcc/x86_64-redhat-linux/4.4.7:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64:/lib/../lib64:/usr/lib/../lib64:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../..:/shared/clang/Release+Asserts/bin/../lib:/lib:/usr/lib ``` As you can see, clang is picking up the library search paths from the system default compiler (gcc 4.4), rather than the search paths with which clang was built (gcc 4.9). Using `LD_LIBRARY_PATH` is not an option -- our build scripts deliberately unset `LD_LIBRARY_PATH` before linking to ensure that only specifically named library paths are searched. I'm attaching the config.log, which appears to show clang picking up gcc 4.9.0 for the build (as expected). The command used to build clang is: ``` make clean distclean;CFLAGS="-std=gnu89" ../llvm/configure --prefix=/build/share/clang --enable-optimized --enable-targets=x86_64 && make ``` At this point I'm not necessarily looking for a fix, but an indication whether this is a legitimate problem with clang, or possibly a problem on our end. Thanks!

llvm / llvm-project

clang doesnt use library search path that it was built with #20884

Extended Description