clang invocations with multiple -arch flags are not cached

[kastiglione]

For some build configs, sccache errors when compiling llvm compiler-rt. The error is:

clang-10: error: cannot use 'c++-cpp-output' output with multiple -arch options

It's true, clang's invocation has two -arch flags (-arch x86_64 -arch x86_64h). I am new to sccache, but from my brief investigation, this seems to break the strategy of preprocessing before compiling. Maybe sccache will need to run the preprocessor once for each arch.

[froydnj]

Maybe sccache will need to run the preprocessor once for each arch.

Yes, this would require some surgery on how hash keys are computed. The other alternative (probably not great for your purposes) is to give up when multiple -arch options are seen, and accept that we can't cache such compilations.

[kastiglione]

Yes I have a wrapper, sccachew, that skips sccache when the args have more than one arch value.

[mathstuf]

Does the compiler get grumpy if one excises all but one -arch flag, but leaves in other -X$arch flags for arch-specific flag communication? If so, the -X$arch flags would also need filtered out before preprocessing as well.

[glandium]

other -X$arch flags

What -X$arch flags? Do you have examples?

[mathstuf]

You can send arguments to just specific arch TU compilations. For example, -Xx86_64 -mno-avx512 since the -mno-avx512 makes no sense on an aarch64 compilation. I forget the exact spelling of -Xx86_64 right now, but it exists.

[glandium]

I was thinking that's what it is about, but I couldn't find what the -X$arch was supposed to be, because -Xx86_64 or -Xarm64 definitely don't work, and clang --help doesn't seem to list such a flag (but then, it also doesn't list -arch).

[mathstuf]

Seems that it is -Xarch_x86_64 and such: https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-xarch-arg1

[glandium]

Nice find. Thanks.

[glandium]

Re-opening and re-purposing to handle this without a "cannotcache" in the future. For that to happen, we'd need to preprocess twice, once with each -arch, taking into account the extra -Xarch$arch flags per the previous comments.

[alcroito]

Compilation with latest 0.4.1 version is now broken if -Xarch flags are used.

$ cat lib.c
void foo() {

# Broken
$ sccache clang -std=c11 -arch arm64 -arch x86_64 -Xarch_arm64 -Ifake -Xarch_x86_64 -Ifake -o foo.c.o -c lib.c
clang: error: invalid Xarch argument: '-Xarch_arm64 -Xarch_x86_64', options requiring arguments are unsupported

# Works
$ clang -std=c11 -arch arm64 -arch x86_64 -Xarch_arm64 -Ifake -Xarch_x86_64 -Ifake -o foo.c.o -c lib.c

The broken command gets rewrritten to

"/usr/bin/clang" "-x" "c" "-E" "-P" "lib.c" "-Ifake" "-Ifake" "-std=c11" "-Xarch_arm64" "-Xarch_x86_64" "-D__arm64__=1" "-D__x86_64__=1"

but the -Ifake flags need to appear after -Xarch.

I assume this is due to https://github.com/mozilla/sccache/pull/1467

Previously the compilation didn't fail, and sccache just didn't cache the result.

[Tippellh07]

Compilation with latest 0.4.1 version is now broken if -Xarch flags are used.

$ cat lib.c
void foo() {

# Broken
$ sccache clang -std=c11 -arch arm64 -arch x86_64 -Xarch_arm64 -Ifake -Xarch_x86_64 -Ifake -o foo.c.o -c lib.c
clang: error: invalid Xarch argument: '-Xarch_arm64 -Xarch_x86_64', options requiring arguments are unsupported

# Works
$ clang -std=c11 -arch arm64 -arch x86_64 -Xarch_arm64 -Ifake -Xarch_x86_64 -Ifake -o foo.c.o -c lib.c

The broken command gets rewrritten to

"/usr/bin/clang" "-x" "c" "-E" "-P" "lib.c" "-Ifake" "-Ifake" "-std=c11" "-Xarch_arm64" "-Xarch_x86_64" "-D__arm64__=1" "-D__x86_64__=1"

but the -Ifake flags need to appear after -Xarch.

I assume this is due to #1467

Previously the compilation didn't fail, and sccache just didn't cache the result.

A couple of possible actions that I can see to help with the problem @alcroito is facing:

1. Update the changes in #1467 to preserve ordering as was in the initial command. Not 100% sure how this would best be done without fairly large changes.
2. If the -Xarch flags aren't needed for the preprocessor step (not sure on this), then add them to the special casing of arch flags that was done in #1467.
3. Make the command rewriting that was added in #1467 confirgurable so that it can be turned off via a config option. This would also be useful for if any other cases are encountered where the rewriting of the command for the preprocessor causes issues so is probably worth doing even if 1/2 solves this particular issue

[mathstuf]

If the -Xarch flags aren't needed for the preprocessor step (not sure on this), then add them to the special casing of arch flags that was done

In general, yes. These flags can change things that __has_builtin can query or the state of intrinsic-detecting preprocessor macros (e.g., "is NEON available?").