search

ClangBuiltLinux / linux

Linux kernel source tree
Other
242 stars 14 forks source link

Mainline kernel build fails early on ThunderX2 #656

Closed ikitayama closed 3 years ago

ikitayama commented 5 years ago

Hi folks,

  1. what kernel version (mainline, LTS) - Mainline
  2. any config in particular is related - defconfig is selected
  3. the precise error message 
    [r1050@apollo linux]$ make CC=clang
scripts/kconfig/conf  --syncconfig Kconfig
arch/arm64/Makefile:54: CROSS_COMPILE_COMPAT is clang, the compat vDSO will not be built
UPD     include/config/kernel.release
UPD     include/generated/utsrelease.h
CC      scripts/mod/empty.o
MKELF   scripts/mod/elfconfig.h
HOSTCC  scripts/mod/modpost.o
CC      scripts/mod/devicetable-offsets.s
In file included from scripts/mod/devicetable-offsets.c:3:
In file included from ./include/linux/mod_devicetable.h:13:
In file included from ./include/linux/uuid.h:12:
./include/linux/string.h:9:10: fatal error: 'stdarg.h' file not found
#include <stdarg.h>
     ^~~~~~~~~~
1 error generated.
make[2]: *** [scripts/mod/devicetable-offsets.s] Error 1
make[1]: *** [prepare0] Error 2
make: *** [sub-make] Error 2
  4. what version of LLD you're using (as we may have already observed the error and fixed it in a newer version). - I am going to use the latest LLD from Trunk of LLVM LLD project repo.
ikitayama commented 5 years ago

I'm reporting here as Clang on a ppc64le machine (POWER8) at JSC is not affected with this header issue at all.

ihalip commented 5 years ago

You can run make V=1 or look at scripts/mod/.devicetable-offsets.s.cmd to see how the compiler is invoked. You can then copy/paste the entire line and add -v to get more diagnostics, such as these:

ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/llvm-8/lib/clang/8.0.0/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

On my Ubuntu workstation, that header seems to be provided by the libclang-common-VER-dev package:

$ apt-file search /usr/lib/llvm-8/lib/clang/8.0.0/include/stdarg.h
libclang-common-8-dev: /usr/lib/llvm-8/lib/clang/8.0.0/include/stdarg.h
ikitayama commented 5 years ago

I have Clang installed non-standard location as I build Trunk as a normal user daily, in that case is there an environmental variable to set prior to invoking a kernel build job?

On Mon, Sep 16, 2019 at 15:04 Ilie Halip notifications@github.com wrote:

You can run make V=1 or look at scripts/mod/.devicetable-offsets.s.cmd to see how the compiler is invoked. You can then copy/paste the entire line and add -v to get more diagnostics, such as these:

ignoring nonexistent directory "/include"

include "..." search starts here:

include <...> search starts here:

/usr/local/include /usr/lib/llvm-8/lib/clang/8.0.0/include /usr/include/x86_64-linux-gnu /usr/include End of search list.

On my Ubuntu workstation, that header seems to be provided by the libclang-VER-common-dev package:

$ apt-file search /usr/lib/llvm-8/lib/clang/8.0.0/include/stdarg.h libclang-common-8-dev: /usr/lib/llvm-8/lib/clang/8.0.0/include/stdarg.h

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ClangBuiltLinux/linux/issues/656?email_source=notifications&email_token=AAHLBZZH74PJZFXQJK65SILQJ4OWHA5CNFSM4IWYVAP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6YFUBA#issuecomment-531651076, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHLBZ2LP65GYBTBW3KZDKLQJ4OWHANCNFSM4IWYVAPQ .

nickdesaulniers commented 5 years ago

I have Clang installed non-standard location

I think 785f11aa595bc3d4e74096cbd598ada54ecc0d81 and ef8c4ed9db802 were made to support this use case.

I am going to use the latest LLD from Trunk of LLVM LLD project repo. The invocation [r1050@apollo linux]$ make CC=clang alludes to LLD not being used, though Clang can be configured to default to LLD. LLD at this point seems to be a red herring; not related.

@ikitayama what version of clang are you using? Can it build hello world? scripts/mod/devicetable-offsets.s is a host utility, so stdarg.h should be the host's stdarg.h IIUC. stdarg.h is used for declaring va_start and friends for variadic function support.

I recommend first trying to compile the example from man va_start on your system with that version of clang. It seems like there's something going wrong or missing on your host that would prevent the building of such host side utilities.

ikitayama commented 5 years ago

@nickdesaulniers first of all, as working on ThunderX2, I am not doing cross-builds. Here's the Clang version:

$ clang --version
clang version 10.0.0 (/fefs/home/r1050/projects/llvm-project/clang 10151f661854e3ee4922662f1d0f62b327cbfa8c)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /fefs/home/r1050/opt/clang/current/bin

... and yes I can compile a hello world program without an issue.

nathanchance commented 5 years ago

packet.com has Thunder X systems available for spot usage, I will try to test/investigate this tonight or tomorrow.

EDIT: Thunder X 2x, not Thunder X2. Shouldn't matter, this is most likely a native arm64 issue, not related to a certain processor.

ikitayama commented 5 years ago

packet.com has Thunder X systems available for spot usage, I will try to test/investigate this tonight or tomorrow.

EDIT: Thunder X 2x, not Thunder X2. Shouldn't matter, this is most likely a native arm64 issue, not related to a certain processor.

Correct.

ikitayama commented 5 years ago

Using Arm Clang (part of HPC Compiler Suite) does not solve the issue either.

nathanchance commented 5 years ago

I cannot reproduce this on Packet's Ampere system on Ubuntu 18.04.

My environment setup script (for this experiment, I did not install clang): https://github.com/nathanchance/scripts/blob/cb7b1b2c2bae6940b11d9db1f399802bb213cc3b/env/generic

$ git clone git://github.com/ClangBuiltLinux/tc-build && cd tc-build && ./build-llvm.py --build-stage1-only

========================
== Checking CC and LD ==
========================

CC: /usr/bin/aarch64-linux-gnu-gcc-7
CXX: /usr/bin/g++
LD: /usr/bin/ld.gold

===========================
== Checking dependencies ==
===========================

/usr/bin/cmake
/usr/bin/curl
/home/nathan/usr/bin/git
/usr/bin/ninja
...

$ cd .. && git clone git://github.com/torvalds/linux && cd linux

$ PATH=/home/nathan/tc-build/build/llvm/stage1/bin:${PATH} make -j$(nproc) CC=clang KCFLAGS=-Wno-implicit-fallthrough LD=ld.lld defconfig vmlinux
...
  AR      drivers/built-in.a
  GEN     .version
  CHK     include/generated/compile.h
  LD      vmlinux.o
  MODPOST vmlinux.o
  MODINFO modules.builtin.modinfo
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map

V=1 shows that lib/clang/10.0.0/include is added to the system include search path via -isystem which is where stdarg.h is available:

clang -Wp,-MD,scripts/mod/.devicetable-offsets.s.d -nostdinc -isystem /home/nathan/tc-build/build/llvm/stage1/lib/clang/10.0.0/include -I../arch/arm64/include -I./arch/arm64/include/generated -I../include -I./include -I../arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi -I../include/uapi -I./include/generated/uapi -include ../include/linux/kconfig.h -include ../include/linux/compiler_types.h -D__KERNEL__ -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3 -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 -no-integrated-as -Werror=unknown-warning-option -mgeneral-regs-only -DCONFIG_AS_LSE=1 -DCONFIG_CC_HAS_K_CONSTRAINT=1 -fno-asynchronous-unwind-tables -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -Wdeclaration-after-statement -Wimplicit-fallthrough -Wvla -Wno-pointer-sign -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-implicit-fallthrough -I ../scripts/mod -I ./scripts/mod -DKBUILD_BASENAME='"devicetable_offsets"' -DKBUILD_MODNAME='"devicetable_offsets"'  -fverbose-asm -S -o scripts/mod/devicetable-offsets.s ../scripts/mod/devicetable-offsets.c
ikitayama commented 5 years ago

Nathan, Can you do the same if possible, on CentOS?

On Thu, Sep 19, 2019 at 3:54 Nathan Chancellor notifications@github.com wrote:

I cannot reproduce this on Packet's Ampere system https://www.packet.com/cloud/servers/c2-large-arm/ on Ubuntu 18.04.

My environment setup script (for this experiment, I did not install clang):

https://github.com/nathanchance/scripts/blob/cb7b1b2c2bae6940b11d9db1f399802bb213cc3b/env/generic

$ git clone git://github.com/ClangBuiltLinux/tc-build && cd tc-build && ./build-llvm.py --build-stage1-only

======================== == Checking CC and LD ==

CC: /usr/bin/aarch64-linux-gnu-gcc-7 CXX: /usr/bin/g++ LD: /usr/bin/ld.gold

=========================== == Checking dependencies ==

/usr/bin/cmake /usr/bin/curl /home/nathan/usr/bin/git /usr/bin/ninja ...

$ cd .. && git clone git://github.com/torvalds/linux && cd linux

$ PATH=/home/nathan/tc-build/build/llvm/stage1/bin:${PATH} make -j$(nproc) CC=clang KCFLAGS=-Wno-implicit-fallthrough LD=ld.lld defconfig vmlinux ... AR drivers/built-in.a GEN .version CHK include/generated/compile.h LD vmlinux.o MODPOST vmlinux.o MODINFO modules.builtin.modinfo KSYM .tmp_kallsyms1.o KSYM .tmp_kallsyms2.o LD vmlinux SORTEX vmlinux SYSMAP System.map

V=1 shows that lib/clang/10.0.0/include is added to the system include search path via -isystem which is where stdarg.h is available:

clang -Wp,-MD,scripts/mod/.devicetable-offsets.s.d -nostdinc -isystem /home/nathan/tc-build/build/llvm/stage1/lib/clang/10.0.0/include -I../arch/arm64/include -I./arch/arm64/include/generated -I../include -I./include -I../arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi -I../include/uapi -I./include/generated/uapi -include ../include/linux/kconfig.h -include ../include/linux/compiler_types.h -DKERNEL -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3 -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 -no-integrated-as -Werror=unknown-warning-option -mgeneral-regs-only -DCONFIG_AS_LSE=1 -DCONFIG_CC_HAS_K_CONSTRAINT=1 -fno-asynchronous-unwind-tables -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -Wdeclaration-after-statement -Wimplicit-fallthrough -Wvla -Wno-pointer-sign -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-implicit-fallthrough -I ../scripts/mod -I ./scripts/mod -DKBUILD_BASENAME='"devicetable_offsets"' -DKBUILD_MODNAME='"devicetable_offsets"' -fverbose-asm -S -o scripts/mod/devicetable-offsets.s ../scripts/mod/devicetable-offsets.c

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ClangBuiltLinux/linux/issues/656?email_source=notifications&email_token=AAHLBZ4A6FVPIPZ7T2PZ5BLQKJ2NHA5CNFSM4IWYVAP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7BCZDA#issuecomment-532819084, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHLBZ6IULBFLBKE3CS4CR3QKJ2NHANCNFSM4IWYVAPQ .

nathanchance commented 5 years ago

Sure, I will try to get to it tonight.

nathanchance commented 5 years ago

@ikitayama everything is fine on CentOS as well, although I had to do a lot more environment hacking because of how outdated CentOS is... I'd recommend giving the tc-build script a shot to see if it is something up with the way that you have configured LLVM. Alternatively, try make CC="clang -v" V=1 like @ihalip suggested to see if lib/clang/10.0.0/include is getting added to the include paths properly.

ikitayama commented 5 years ago

@nathanchance thanks! Our machine, HPE Apollo is backed by Red Hat Enterprise Linux Server release 7.5 (Maipo), and it doesn't seem to contain Python 3, so I can not try your script at this moment.

ikitayama commented 5 years ago 
$ clang -Wp,-MD,scripts/mod/.devicetable-offsets.s.d -nostdinc -isystem /opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/include -I./arch/arm64/include -I./arch/arm64/include/generated -I./include -I./arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3 -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 -no-integrated-as -Werror=unknown-warning-option -mgeneral-regs-only -DCONFIG_AS_LSE=1 -DCONFIG_CC_HAS_K_CONSTRAINT=1 -fno-asynchronous-unwind-tables -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -pg -Wdeclaration-after-statement -Wimplicit-fallthrough -Wvla -Wno-pointer-sign -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-format -Wno-sign-compare -Wno-format-zero-length -fsanitize=kernel-address -mllvm -asan-mapping-offset=0xdffffe8000000000 -mllvm -asan-globals=1 -mllvm -asan-instrumentation-with-call-threshold=10000 -mllvm -asan-stack=0 --param asan-instrument-allocas=1 -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=unreachable -fsanitize=signed-integer-overflow -fsanitize=bounds -fsanitize=object-size -fsanitize=bool -fsanitize=enum -fsanitize-coverage=trace-pc -fsanitize-coverage=trace-cmp -DKBUILD_BASENAME='"devicetable_offsets"' -DKBUILD_MODNAME='"devicetable_offsets"'  -fverbose-asm -S -o scripts/mod/devicetable-offsets.s scripts/mod/devicetable-offsets.c
In file included from scripts/mod/devicetable-offsets.c:3:
In file included from ./include/linux/mod_devicetable.h:13:
In file included from ./include/linux/uuid.h:12:
./include/linux/string.h:9:10: fatal error: 'stdarg.h' file not found
#include <stdarg.h>
         ^~~~~~~~~~
1 error generated.
ikitayama commented 5 years ago

GCC reports the include directory where stdarg.h lives 

$ gcc -print-file-name=include
/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/bin/../lib/gcc/aarch64-linux/8.2.0/include

, but locally built trunk Clang does not:

$ clang -print-file-name=include
/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/include
ikitayama commented 5 years ago

Now I am getting lots of these:

./include/linux/mm.h:162:2: warning: unannotated fall-through between switch labels [-Wimplicit-fallthrough]
        case 72:
        ^

Have you guys seen these before on arm64?

nathanchance commented 5 years ago

This is being tracked in #236. It is disabled in -next and will be disabled in the next kbuild pull request (so 5.4-rc1 will have that resolved).

nathanchance commented 5 years ago

GCC reports the include directory where stdarg.h lives

$ gcc -print-file-name=include
/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/bin/../lib/gcc/aarch64-linux/8.2.0/include

, but locally built trunk Clang does not:

$ clang -print-file-name=include
/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/include

This seems like a configuration issue, can you post your clang configure command?

ikitayama commented 5 years ago

This is how I configure clang and other sub-probjects atop of llvm.

cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${now} -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="" -DLLVM_TARGETS_TO_BUILD="AArch64;NVPTX" -DLLVM_ENABLE_PROJECTS="clang;openmp;libcxx;libcxxabi;libunwind;lld" ../llvm
nathanchance commented 5 years ago

Wonder if it could somehow be related to libcxx;libcxxabi, since I don't enable those projects for the kernel...

ikitayama commented 5 years ago

Did not change at all.

On Sat, Sep 21, 2019 at 2:45 AM Nathan Chancellor notifications@github.com wrote:

Wonder if it could somehow be related to libcxx;libcxxabi, since I don't enable those projects for the kernel...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ClangBuiltLinux/linux/issues/656?email_source=notifications&email_token=AAHLBZZ7AQ5U7VHX7ZYUZS3QKUD2JA5CNFSM4IWYVAP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7HNIEQ#issuecomment-533648402, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHLBZ4PWAXUYVCHFX3DZCDQKUD2JANCNFSM4IWYVAPQ .

ikitayama commented 5 years ago

I’ll check out your Clang config/build script and Let you know how it goes on ThindrtX2.

On Sat, Sep 21, 2019 at 7:17 Itaru Kitayama itaru.kitayama@gmail.com wrote:

Did not change at all.

On Sat, Sep 21, 2019 at 2:45 AM Nathan Chancellor < notifications@github.com> wrote:

Wonder if it could somehow be related to libcxx;libcxxabi, since I don't enable those projects for the kernel...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ClangBuiltLinux/linux/issues/656?email_source=notifications&email_token=AAHLBZZ7AQ5U7VHX7ZYUZS3QKUD2JA5CNFSM4IWYVAP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7HNIEQ#issuecomment-533648402, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHLBZ4PWAXUYVCHFX3DZCDQKUD2JANCNFSM4IWYVAPQ .

nickdesaulniers commented 5 years ago

or -DCMAKE_INSTALL_PREFIX=$HOME/opt/clang/${now} (not sure how that would affect the resulting build).

@ikitayama my CMake invocation usually looks like:

cmake .. -DCMAKE_BUILD_TYPE=Release -G Ninja -DCMAKE_C_COMPILER=/android1/android-master/prebuilts/clang/host/linux-x86/clang-r353983c/bin/clang -DCMAKE_CXX_COMPILER=/android1/android-master/prebuilts/clang/host/linux-x86/clang-r353983c/bin/clang++ -DLLVM_ENABLE_LLD=ON -DLLVM_ENABLE_PROJECTS="clang;lld" -DLLVM_TARGETS_TO_BUILD="AArch64;ARM;X86"
ikitayama commented 5 years ago

Our Apollo server will be offline from this weekend for about a month (expected). I will try to re-visit this Issue when it comes back on-line. So, please keep this Issue open.

ikitayama commented 5 years ago

By the way, I heard Travis now can offer Arm, would you consider deploying it on that arch?

nathanchance commented 5 years ago

https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support

I would like to eventually do this. The main issue is apt.llvm.org only does x86_64 right now, meaning we would need to build tip of tree LLVM ourselves but I am not sure any of us have consistent access to arm64 hardware and cross compiling LLVM for arm64 does not exactly sound fun :(

ikitayama commented 5 years ago

Our HPE Apollo is back after some downtime. I am still seeing the same error when trying to build kernel with my trunk Clang.

ikitayama commented 4 years ago

Hi, Turned out my local build returns some irrelevant directory using a full path returned by 

clang -print-file-name=include

. So, I am in the meantime setting it manually in the Makefile. Do you guys know how to set it properly?

ikitayama commented 4 years ago

Now I am able to build the kernel with trunk Clang 10ish.

ikitayama commented 4 years ago 
$ clang -print-search-dirs
programs: =/fefs/home/r1050/opt/clang/current/bin:/fefs/home/r1050/opt/clang/201912120719/bin:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../../../aarch64-linux-gnu/bin
libraries: =/fefs/home/r1050/opt/clang/201912120719/lib/clang/10.0.0:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../../../lib64:/lib/../lib64:/usr/lib/../lib64:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../../../aarch64-linux-gnu/lib:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../..:/fefs/home/r1050/opt/clang/201912120719/bin/../lib:/lib:/usr/lib
$ clang -print-file-name=include
/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/include

I am puzzled, but can someone see what's wrong with my build configuration?

nickdesaulniers commented 4 years ago 
➜  ~ clang -print-search-dirs
programs: =/android0/llvm-project/llvm/build/bin:/usr/lib/gcc/x86_64-linux-gnu/8/../../../../x86_64-linux-gnu/bin
libraries: =/android0/llvm-project/llvm/build/lib/clang/10.0.0:/usr/lib/gcc/x86_64-linux-gnu/8:/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu:/usr/lib/gcc/x86_64-linux-gnu/8/../../../../lib64:/lib/x86_64-linux-gnu:/lib/../lib64:/usr/lib/x86_64-linux-gnu:/usr/lib/../lib64:/usr/lib/x86_64-linux-gnu/../../lib64:/usr/lib/gcc/x86_64-linux-gnu/8/../../..:/android0/llvm-project/llvm/build/bin/../lib:/lib:/usr/lib
➜  ~ clang -print-file-name=include
/android0/llvm-project/llvm/build/lib/clang/10.0.0/include

/android0/llvm-project/ is my llvm tree.

ikitayama commented 4 years ago

Nick, is it also complicated Clang’s library finding scheme?

ikitayama commented 4 years ago

The directory, /fefs/home/r1050/opt/clang/201912120719/lib/clang/10.0.0, does exist, but Clang reports with the option -print-file-name=include that picked up the second one from the "libraries" list of directoeis.

$ clang -print-search-dirs
programs: =/fefs/home/r1050/opt/clang/current/bin:/fefs/home/r1050/opt/clang/201912120719/bin:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../../../aarch64-linux-gnu/bin
libraries: =/fefs/home/r1050/opt/clang/201912120719/lib/clang/10.0.0:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../../../lib64:/lib/../lib64:/usr/lib/../lib64:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../../../aarch64-linux-gnu/lib:/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/lib/gcc/aarch64-linux-gnu/8.2.0/../../..:/fefs/home/r1050/opt/clang/201912120719/bin/../lib:/lib:/usr/lib
$ clang -print-file-name=include
/opt/arm/gcc-8.2.0_Generic-AArch64_RHEL-7_aarch64-linux/include
nickdesaulniers commented 3 years ago

Sorry, I don't think any contributors have aarch64 hosts yet to begin to attempt to reproduce. Perhaps we will have some M1 macbooks running linux soon to test with. Closing for now.