yrnkrn / zapcc

zapcc is a caching C++ compiler based on clang, designed to perform faster compilations
Other
1.25k stars 61 forks source link

Zapcc gets c++ include directories wrong #29

Closed ypnos closed 4 years ago

ypnos commented 5 years ago

When I try to compile a random C++ file without any specific command line options I get this output:

# zapcc -v foo.cpp
clang version 5.0.0 (trunk) (/srcdest/zapcc 35fa2aed76fffa222db340dcb9257f66dc858d42) built on Jun 30 2018 18:44:01
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /bin
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Selected GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/usr/bin/zapcc" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -v -dwarf-column-info -debugger-tuning=gdb -resource-dir /usr/lib/clang/5.0.0 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/x86_64-pc-linux-gnu -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/backward -internal-isystem /usr/local/include -internal-isystem /usr/lib/clang/5.0.0/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /home/ypnos/fun/hlwm -ferror-limit 5 -fmessage-length 318 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/main-e9d804.o -x c++ src/main.cpp
ignoring nonexistent directory "/include/c++/8.2.1"
ignoring nonexistent directory "/include/c++/8.2.1/x86_64-pc-linux-gnu"
ignoring nonexistent directory "/include/c++/8.2.1/backward"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/clang/5.0.0/include
 /usr/include
End of search list.
ignoring nonexistent directory "/include/c++/8.2.1"
ignoring nonexistent directory "/include/c++/8.2.1/x86_64-pc-linux-gnu"
ignoring nonexistent directory "/include/c++/8.2.1/backward"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/clang/5.0.0/include
 /usr/include
End of search list.

The compiler tries to use /include/c++/8.2.1, when it should instead use /usr/include/c++/8.2.1. A workaround for this problem is to call /usr/bin/zapcc directly. clang (version 7) does not have this problem because it includes /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1 instead.

This is an up-to-date Arch Linux system. Note that in Arch Linux, /bin is a symlink to /usr/bin. This is not out of the ordinary, several distributions do this and another major distribution recently announced it will do this in the future as well.

yrnkrn commented 5 years ago

Does this happens with zapcc built from latest commit here?

ypnos commented 5 years ago

Yes, please refer to the output of both zapcc and clang below:

ypnos@void ~/f/hlwm> /bin/zapcc -v src/main.cpp                                                                                                                                                                                                                                                                          01:04
zapcc clang version 7.0.0 (https://github.com/yrnkrn/zapcc) (/home/ypnos/.cache/pikaur/build/zapcc-git/zapcc 9bd04e7618bafaf6d938b0aed5837c1c5b312645) (based on LLVM 7.0.0svn)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /bin
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Selected GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/usr/bin/zapcc" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /usr/lib/clang/7.0.0 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/x86_64-pc-linux-gnu -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/backward -internal-isystem /usr/local/include -internal-isystem /usr/lib/clang/7.0.0/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /home/ypnos/fun/hlwm -ferror-limit 5 -fmessage-length 318 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/main-15f9d2.o -x c++ src/main.cpp
ignoring nonexistent directory "/include/c++/8.2.1"
ignoring nonexistent directory "/include/c++/8.2.1/x86_64-pc-linux-gnu"
ignoring nonexistent directory "/include/c++/8.2.1/backward"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/clang/7.0.0/include
 /usr/include
End of search list.
ignoring nonexistent directory "/include/c++/8.2.1"
ignoring nonexistent directory "/include/c++/8.2.1/x86_64-pc-linux-gnu"
ignoring nonexistent directory "/include/c++/8.2.1/backward"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/clang/7.0.0/include
 /usr/include
End of search list.
clang -cc1 version 5.0.0 based upon LLVM 5.0.0git-35fa2aed default target x86_64-unknown-linux-gnu
In file included from /home/ypnos/fun/hlwm/src/main.cpp:1:
In file included from /home/ypnos/fun/hlwm/src/root.h:4:
/home/ypnos/fun/hlwm/src/types.h:4:10: fatal error: 'vector' file not found
#include <vector>
         ^~~~~~~~
1 error generated.
ypnos@void ~/f/hlwm> /bin/clang -v src/main.cpp                                                                                                                                                                                                                                                                        ⚡ 11:28
clang version 7.0.0 (tags/RELEASE_700/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /bin
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/7.3.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Selected GCC installation: /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/usr/bin/clang-7" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /usr/lib/clang/7.0.0 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1 -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/x86_64-pc-linux-gnu -internal-isystem /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/backward -internal-isystem /usr/local/include -internal-isystem /usr/lib/clang/7.0.0/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /home/ypnos/fun/hlwm -ferror-limit 19 -fmessage-length 318 -stack-protector 2 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/main-22a108.o -x c++ src/main.cpp -faddrsig
clang -cc1 version 7.0.0 based upon LLVM 7.0.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1
 /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/x86_64-pc-linux-gnu
 /bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/backward
 /usr/local/include
 /usr/lib/clang/7.0.0/include
 /usr/include
End of search list.

Btw. I don't understand why it says clang -cc1 version 5.0.0 in the output of zapcc where it should be 7.0?

yrnkrn commented 5 years ago

It seems there is a mixup of zapcc versions installed:

# zapcc -v foo.cpp
clang version 5.0.0 (trunk) (/srcdest/zapcc 35fa2aed76fffa222db340dcb9257f66dc858d42) built on Jun 30 2018 18:44:01
Target: x86_64-unknown-linux-gnu
Thread model: posix

ypnos@void ~/f/hlwm> /bin/zapcc -v src/main.cpp                                                                                                                                                                                                                                                                          
zapcc clang version 7.0.0 (https://github.com/yrnkrn/zapcc) (/home/ypnos/.cache/pikaur/build/zapcc-git/zapcc 9bd04e7618bafaf6d938b0aed5837c1c5b312645) (based on LLVM 7.0.0svn)
Target: x86_64-unknown-linux-gnu

also likely that there are two versions of clang installed, maybe both in /bin. This can cause a mess.

Try to build & install the git version of zapcc in a new, local directory and run it from there.

ypnos commented 5 years ago

No, there is only one version installed. I removed the old and built+installed the new one after you asked about current git. Should have explicitely mentioned it, sorry for the confusion.

There is also only one version of clang installed which is 7.0.0.

yrnkrn commented 5 years ago

OK. This is probably a bug fixed in clang final 7.0.0 released on 2018-09-19, as zapcc was last updated from clang on 2018-02-13 (clang r325000). Patches welcome...

xlicz commented 5 years ago

Hi, I have used zapcc for a long time, it works great! I'd like to do some contribution to this project. I plan to merge newer clang/llvm code to zapcc to get benefits from them such as bug fixes, newer standard support and more compile time check. Are there some merge instructions or tips for reference to make the merge process smooth and correct?

yrnkrn commented 5 years ago

It may be easier to merge in stages, catching up one month or so at a time. Make sure all tests are passing. Thanks!

On Fri, Jul 26, 2019 at 10:06 PM Light Wings notifications@github.com wrote:

Hi, I have used zapcc for a long time, it works great! I'd like to do some contribution to this project. I plan to merge newer clang/llvm code to zapcc to get benefits from them such as bug fixes, newer standard support and more compile time check. Are there some merge instructions or tips for reference to make the merge process smooth and correct?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yrnkrn/zapcc/issues/29?email_source=notifications&email_token=ACAKBFK4UCO3DS354KGX4RLQBNDJPA5CNFSM4GGEZM2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD25OT7Y#issuecomment-515566079, or mute the thread https://github.com/notifications/unsubscribe-auth/ACAKBFKABVMM7RFZCJCXX7DQBNDJPANCNFSM4GGEZM2A .

xlicz commented 5 years ago

Here is the test result for the original zapcc code:

Expected Passes : 35165 Expected Failures : 162 Unsupported Tests : 982 Unexpected Failures: 5

There are 5 test failsures. Is this an expected behavior?

Also, after some fail trials, I found out that I'm just a newbie to work on llvm-like project and there are huge difference between zapcc and LLVM32500 code . It seems that you use an easy to follow process to merge llvm code into zapcc. Would you mind supply me some detail document links so that I can have the merge done? And it would be even better if you add merge instructions to the readme file so any other people can also follow it to merge LLVM into zapcc. Thanks!

yrnkrn commented 5 years ago

Ideally there would be zero failures but if you can keep the number of failures the same its OK. I don't have any easy process for updating. It's a manual operation requiring in-depth knowledge of LLVM, clang & zapcc. May be better to start learning the LLVM & clang projects, committing patches to these projects before moving to merge with zapcc.

On Sat, Jul 27, 2019 at 11:21 AM Light Wings notifications@github.com wrote:

Here is the test result for the original zapcc code:

Expected Passes : 35165 Expected Failures : 162 Unsupported Tests : 982 Unexpected Failures: 5

There are 5 test failsures. Does this an expected behavior?

Also, after some fail trials, I found out that I'm just a newbie to work on llvm-like project. It seems that you use an easy to follow process to merge llvm code into zapcc. Would you mind supply me some detail document links so that I can have the merge done? Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yrnkrn/zapcc/issues/29?email_source=notifications&email_token=ACAKBFNG4M6HQBTROJD2XSTQBQAQVA5CNFSM4GGEZM2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD26G6UQ#issuecomment-515665746, or mute the thread https://github.com/notifications/unsubscribe-auth/ACAKBFJDETFJLALQINQFGP3QBQAQVANCNFSM4GGEZM2A .

xlicz commented 5 years ago

Thanks for your reply! After struggling the whole afternoon and night I reach the same conclusion you provide. :(

Apart from the code in

tools/zapcc
tools/zapccs
tools/clang/test/zapcc

There are also many file change in many other files, such as

zapcc/lib/Transforms/Utils/CMakeLists.txt
zapcc/lib/IR/CMakeLists.txt
zapcc/lib/CodeGen/AsmPrinter/CMakeLists.txt
zapcc/lib/Analysis/CMakeLists.txt
zapcc/tools/clang/lib/Sema/CMakeLists.txt
zapcc/tools/clang/lib/CodeGen/CMakeLists.txt
zapcc/tools/clang/lib/AST/CMakeLists.txt
zapcc/tools/clang/tools/driver/CMakeLists.txt
zapcc/tools/clang/test/CodeGen/CMakeLists.txt
zapcc/lib/Transforms/Utils/InlineFunction.cpp
zapcc/lib/IR/Value.cpp
zapcc/lib/IR/Module.cpp
zapcc/lib/Support/Unix/Program.inc
zapcc/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
zapcc/lib/Analysis/ConstantFolding.cpp
zapcc/tools/clang/include/clang/Driver/Driver.h
zapcc/tools/clang/include/clang/Driver/Options.td
zapcc/tools/clang/include/clang/Sema/Sema.h
zapcc/tools/clang/lib/Sema/SemaDecl.cpp
zapcc/tools/clang/lib/Sema/SemaOverload.cpp
zapcc/tools/clang/lib/CodeGen/ItaniumCXXABI.cpp
zapcc/tools/clang/lib/CodeGen/CodeGenModule.h
zapcc/tools/clang/lib/AST/Decl.cpp
zapcc/tools/clang/tools/driver/driver.cpp
zapcc/tools/clang/test/CodeGen/xcore-stringtype.c
zapcc/tools/clang/CMakeLists.txt
zapcc/utils/lit/lit/TestingConfig.py
zapcc/utils/lit/lit/llvm/config.py

So, when the code changes in Clang, it seems a nightmare to merge LLVM code into zapcc, which can only be done manually, which require both in-depth knowledge of LLVM, clang & zapcc and strong carefulness.

I'm also curious to know the consideration of not maintaining zapcc as a branch of llvm & clang so that we can get patches from them easily. Thanks!

yrnkrn commented 4 years ago

Closing for now.