Native compiler does not search for musl include path

[concatime]

Hi. When building a native compiler using a cross one, I’ve found it does not search for /include, where musl installs its headers.

Step to reproduce

Create a config.mak with:

TARGET := x86_64-unknown-linux-musl

ifneq ($(NATIVE),)
    OUTPUT := /foo
    # stackoverflow.com/a/324782
    TOP := $(abspath $(dir $(lastword $(MAKEFILE_LIST))))
    PATH := $(TOP)/output/bin:$(PATH)
endif

COMMON_CONFIG += --disable-nls
# https://github.com/richfelker/musl-cross-make/issues/97
GCC_CONFIG += --disable-multiarch

Build the cross and native compilers:

make
make install
make NATIVE=1
sudo make NATIVE=1 install

Build a static posix shell (dash):

./configure --enable-static
make
sudo cp src/dash /foo/bin/sh

Finally:

sudo env -i /sbin/chroot /foo
printf '#include <stdio.h>\nint main(void){puts("hi");}\n' | gcc -v -x c -

Here is the output:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=../libexec/gcc/x86_64-unknown-linux-musl/9.2.0/lto-wrapper
Target: x86_64-unknown-linux-musl
Configured with: ../src_gcc/configure --enable-languages=c,c++ --disable-nls --disable-multiarch --disable-bootstrap --disable-assembly --disable-werror --target=x86_64-unknown-linux-musl --prefix= --libdir=/lib --disable-multilib --with-sysroot=/ --enable-tls --disable-libmudflap --disable-libsanitizer --disable-gnu-indirect-function --disable-libmpx --enable-libstdcxx-time=rt --build=x86_64-pc-linux-gnu --host=x86_64-unknown-linux-musl
Thread model: posix
gcc version 9.2.0 (GCC)
COLLECT_GCC_OPTIONS='-v' '-mtune=generic' '-march=x86-64'
 ../libexec/gcc/x86_64-unknown-linux-musl/9.2.0/cc1 -quiet -v -iprefix ../lib/gcc/x86_64-unknown-linux-musl/9.2.0/ -isysroot ../ - -quiet -dumpbase - -mtune=generic -march=x86-64 -auxbase - -version -o ./ccLGmeLJ.s
GNU C17 (GCC) version 9.2.0 (x86_64-unknown-linux-musl)
        compiled by GNU C version 9.2.0, GMP version 6.1.2, MPFR version 4.0.2, MPC version 1.1.0, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "../lib/gcc/x86_64-unknown-linux-musl/9.2.0/../../../../x86_64-unknown-linux-musl/include"
ignoring nonexistent directory "../usr/local/include"
ignoring nonexistent directory "../lib/gcc/../../lib/gcc/x86_64-unknown-linux-musl/9.2.0/../../../../x86_64-unknown-linux-musl/include"
ignoring nonexistent directory "../usr/include"
ignoring duplicate directory "../lib/gcc/../../lib/gcc/x86_64-unknown-linux-musl/9.2.0/include"
#include "..." search starts here:
#include <...> search starts here:
 ../lib/gcc/x86_64-unknown-linux-musl/9.2.0/include
End of search list.
GNU C17 (GCC) version 9.2.0 (x86_64-unknown-linux-musl)
        compiled by GNU C version 9.2.0, GMP version 6.1.2, MPFR version 4.0.2, MPC version 1.1.0, isl version none
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: e90c7fd448a3b4f9ba73cae486deeb01
<stdin>:1:10: fatal error: stdio.h: No such file or directory
compilation terminated.

As you see, neither ../include nor /include is in the search path. One quick and dirty fix is to create /foo/usr folder and create a symlink with ln -s ../include /foo/usr/include.

Here is a stripped output of tree -d /foo:

/foo
├── bin
├── include
│   ├── arpa
│   ├── asm
│   ├── asm-generic
│   ├── bits
│   ├── c++
│   │   └── 9.2.0
│   │       ├── […]
│   │       └── x86_64-unknown-linux-musl
│   │           └── […]
│   ├── drm
│   ├── linux
│   │   └── […]
│   ├── mtd
│   ├── net
│   ├── netinet
│   ├── netpacket
│   ├── rdma
│   │   └── hfi
│   ├── scsi
│   │   └── fc
│   ├── sound
│   ├── sys
│   ├── video
│   └── xen
├── lib
│   └── gcc
│       └── x86_64-unknown-linux-musl
│           └── 9.2.0
│               ├── include
│               │   └── ssp
│               ├── include-fixed
│               ├── install-tools
│               │   └── include
│               └── plugin
│                   └── include
│                       └── […]
├── libexec
│   └── gcc
│       └── x86_64-unknown-linux-musl
│           └── 9.2.0
│               ├── install-tools
│               └── plugin
├── share
│   ├── gcc-9.2.0
│   │   └── python
│   │       └── libstdcxx
│   │           └── v6
│   └── man
│       ├── man1
│       └── man7
└── x86_64-unknown-linux-musl
    ├── bin
    └── lib
        └── ldscripts

[concatime]

From https://gcc.gnu.org/install/configure.html:

If you specify the --with-native-system-header-dir=dirname option then the compiler will search that directory within dirname for native system headers rather than the default /usr/include.

[michaelforney]

A patch for this issue is available on the musl mailing list: https://www.openwall.com/lists/musl/2020/03/23/1

[concatime]

Is there a reason why it’s not yet merged?

[richfelker]

See https://www.openwall.com/lists/musl/2020/05/10/1 and https://www.openwall.com/lists/musl/2020/05/10/4

[firasuke]

I was also wondering why this was happening as well, and why the dirty solution to symlink usr to . was needed...

So any updates on the issue? I agree with the addition of --with-native-system-header-dir=/include because it's what's expected by users regardless if it's 100% accurate...