Open alexcrichton opened 6 years ago
Oh, this sucks. :-( gcc has a --print-prog-name=
option that sounds almost useful, but it doesn't seem to do anything useful if you pass it cc or gcc:
luser@eye7:/build$ gcc --print-prog-name=gcc
gcc
luser@eye7:/build$ gcc-5 --print-prog-name=gcc-5
gcc-5
luser@eye7:/build$ gcc --print-prog-name=cc
cc
luser@eye7:/build$ gcc-5 --print-prog-name=cc
cc
clang's implementation of this seems to be more useful:
luser@eye7:/build$ clang --print-prog-name=clang
/usr/bin/clang
luser@eye7:/build$ PATH=/build/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-16.04/bin/:$PATH clang --print-prog-name=clang
/build/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-16.04/bin/clang
gcc's -print-search-dirs
option does seem to include an install:
key which might be usable:
luser@eye7:/build$ gcc -print-search-dirsinstall: /usr/lib/gcc/x86_64-linux-gnu/7/
programs: =/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/bin/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/bin/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/bin/
libraries: =/usr/lib/gcc/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/lib/x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/lib/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/7/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib/:/lib/x86_64-linux-gnu/7/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/7/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../../x86_64-linux-gnu/lib/:/usr/lib/gcc/x86_64-linux-gnu/7/../../../:/lib/:/usr/lib/
luser@eye7:/build$ gcc-5 -print-search-dirs
install: /usr/lib/gcc/x86_64-linux-gnu/5/
programs: =/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/bin/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/bin/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/bin/
libraries: =/usr/lib/gcc/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/lib/x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/lib/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/5/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib/:/lib/x86_64-linux-gnu/5/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/5/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/lib/:/usr/lib/gcc/x86_64-linux-gnu/5/../../../:/lib/:/usr/lib/
With the toolchains in question, do you get useful output from that? If so we could fix sccache to use this as a cache input somehow, at the cost of an additional gcc invocation per compile (but presumably these -print-...
commands are fast). Making it conditional on a heuristic that checks whether the compiler binary actually starts with a shebang might be good enough?
Ah unfortunately for whatever reason the shell scripts for the musl compilers that collided in https://github.com/rust-lang/rust/pull/48647 both don't work with --print-prog-name
:
root@e95d6cbf1fcf:/build# mipsel-openwrt-linux-gcc --print-prog-name=gcc
gcc
root@e95d6cbf1fcf:/build# mipsel-openwrt-linux-gcc --print-prog-name=mipsel-openwrt-linux-gcc
mipsel-openwrt-linux-gcc
root@e95d6cbf1fcf:/build# mips-openwrt-linux-gcc --print-prog-name=mips-openwrt-linux-gcc
mips-openwrt-linux-gcc
root@e95d6cbf1fcf:/build# mips-openwrt-linux-gcc --print-prog-name=gcc
gcc
they do, however, have unique -print-search-dirs
prints!
Making it conditional on a heuristic that checks whether the compiler binary actually starts with a shebang might be good enough?
Yeah that sounds totally plausible to me, and that way if someone wants to skip the extra invocation you could just switch it with a native executable (or something similar)
Fun variant on this: on macOS, the clang
binary is actually a weird find-the-real-clang
indirection thing:
~ ❯❯❯ nm /usr/bin/clang
0000000100000000 T __mh_execute_header
0000000100000f77 T _main
0000000100001018 S _shim_marker
U _xcselect_invoke_xcrun
U dyld_stub_binder
The last modified time on /usr/bin/clang
for me is the 4th of July, which is interesting because a) I'm quite certain I wasn't using this computer on the 4th, b) I just upgraded my clang
this morning. From this I conclude, that for all practical purposes, the "compiler changed" logic is never invoked for the system clang
on macOS.
(For some additional color... I discovered this because my Firefox builds are done with ASAN. On macOS, ASAN has a symbol with a version in it, presumably to prevent incompatibility between the compiler-emitted ASAN bits and the runtime library. The result is that if you try to use the new linker with an old object file, you get an error as a result. This is not a part of upstream ASAN AFAIK.)
Random though after having hit a similar problem again, how about hashing the output of $compiler --version
instead of the binary?
This is sort of similar to https://github.com/mozilla/sccache/issues/87 except for C compilers. First discovered here we had a situation that looked like:
mips-openwrt-linux-gcc
andmipsel-openwrt-linux-gcc
, were used$0
to figure out where to go (and is why the shell script operated differently in the two locations)sccache
theexecutable_digest
part of the hash key was then the same for both compilersAs a consequence this meant that the mips object files (built first) were used for the mipsel compiler, causing our issues!
I'm not really sure what the best way to solve this would be :(