systemz: "error in backend: Unsupported stack frame traversal count"

[9816046d-d175-4de2-95d1-ce389e5190e8]

Bugzilla Link	41424
Version	8.0
OS	Linux
Blocks	llvm/llvm-project#4440
Attachments	[preprocessed kernelhttps://user-images.githubusercontent.com/92601431/143759005-1758b79f-c88b-4249-95d2-0271749bd8b3.gz), reproducer sript for sched_wakeup.c
CC	@nickdesaulniers,@uweigand

Extended Description

I get a build failure for various file when trying to compile the linux kernel for s390 (systemz):

clang-8" "-cc1" "-triple" "s390x-unknown-linux-gnu" "-S" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "trace_sched_wakeup.c" "-mrelocation-model" "static" "-mthread-model" "posix" "-relaxed-aliasing" "-fmath-errno" "-masm-verbose" "-no-integrated-as" "-mconstructor-aliases" "-fuse-init-array" "-target-cpu" "z196" "-mbackchain" "-dwarf-column-info" "-debug-info-kind=limited" "-dwarf-version=4" "-debugger-tuning=gdb" "-momit-leaf-frame-pointer" "-coverage-notes-file" "/git/arm-soc/y2038/obj-s390/kernel/trace/trace_sched_wakeup.gcno" "-nostdsysteminc" "-nobuiltininc" "-sys-header-deps" "-D" "KERNEL" "-D" "CONFIG_AS_CFI_VAL_OFFSET=1" "-D" "CC_USING_FENTRY" "-D" "KBUILD_BASENAME=\"trace_sched_wakeup\"" "-D" "KBUILD_MODNAME=\"trace_sched_wakeup\"" "-O2" "-Wall" "-Wundef" "-Werror=strict-prototypes" "-Wno-trigraphs" "-Werror=implicit-function-declaration" "-Werror=implicit-int" "-Wno-format-security" "-Wno-sign-compare" "-Wno-format-invalid-specifier" "-Wno-gnu" "-Wno-address-of-packed-member" "-Wno-tautological-compare" "-Wno-unused-const-variable" "-Wdeclaration-after-statement" "-Wvla" "-Wno-pointer-sign" "-Werror=date-time" "-Werror=incompatible-pointer-types" "-Wno-initializer-overrides" "-Wno-unused-value" "-Wno-format" "-Wno-sign-compare" "-Wno-format-zero-length" "-std=gnu89" "-fno-dwarf-directory-asm" "-ferror-limit" "19" "-fmessage-length" "132" "-fwrapv" "-fno-signed-char" "-fwchar-type=short" "-fno-signed-wchar" "-fobjc-runtime=gcc" "-fno-common" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-vectorize-loops" "-vectorize-slp" "-x" "c" "trace_sched_wakeup-ff17db.c"

fatal error: error in backend: Unsupported stack frame traversal count

I did not try reducing the test case, but could do so if it's not immediately clear from the output where the problem is.

[uweigand]

In general I'm happy to add features to LLVM where it makes sense to support building the kernel. In this particular case it doesn't seem necessary.

I've just talked to Martin and he confirms that the kernel (also on SystemZ) is moving to using ORC for unwinding, which implies that neither -mkernel-backchain nor __builtin_return_address(n) with n>0 will be used any more.

The only remaining issue is that it would indeed be nicer if we got a front-end warning instead of a back-end internal error. Unfortunately that's a deficiency in clang at the moment (__builtin_return_address is handled by platform-independent code that is not aware of platform-specific limitations). I'll see what I can do about that.

[nickdesaulniers]

Why is it a requirement to build the kernel with LLVM? None of the distros do as far as I know ...

Kind of a non-starter if there's missing features the kernel depends on.

[9816046d-d175-4de2-95d1-ce389e5190e8]

I submitted a kernel patch to avoid the use __builtin_return_address() with clang, see

https://lore.kernel.org/lkml/20190408212648.2407234-10-arnd@arndb.de/

Including that one in the kernel should be sufficient. However, it would be helpful to have a more specific error message here (i.e. including the source code location), and possibly turn it into a warning, to match the gcc behavior when backchains are disabled. On the reduced test case, gcc only warns

test.c:3:9: warning: unsupported argument to '__builtin_return_address' fn1() { fn2(CALLER_ADDR1); } ^~~~~

[9816046d-d175-4de2-95d1-ce389e5190e8]

If this is important, I think we could add the same support as is in GCC today. As in GCC, it would only work reliably if the whole code base is built with -mbackchain.

I understand that even the kernel doesn't really want the stack size overhead involved with -mbackchain, which is why a while back we implemented a special -mkernel-backchain feature in GCC just for the kernel. This isn't implemented in LLVM at all at the moment.

Ok, I see.

Why is it a requirement to build the kernel with LLVM? None of the distros do as far as I know ...

Correct: to my knowledge, there is currently nobody trying to use such a kernel in practice. My motivation was purely for build testing, and making sure that the clang binaries I built for creating ARM kernels work across other architectures as well.

[uweigand]

Just like GCC, LLVM will not allow __builtin_return_address(N) for any N > 0 in the absence of a backchain (because there is no way to retrieve the return address in higher frames -- except by reading DWARF).

GCC does support builtin_return_address(N) for N > 0 if you also build with backchain (-mbackchain). LLVM doesn't do that even that. This is because originally LLVM didn't support -mbackchain at all. While it now does, we didn't change the builtin_return_address implementation.

If this is important, I think we could add the same support as is in GCC today. As in GCC, it would only work reliably if the whole code base is built with -mbackchain.

I understand that even the kernel doesn't really want the stack size overhead involved with -mbackchain, which is why a while back we implemented a special -mkernel-backchain feature in GCC just for the kernel. This isn't implemented in LLVM at all at the moment.

Why is it a requirement to build the kernel with LLVM? None of the distros do as far as I know ...

[9816046d-d175-4de2-95d1-ce389e5190e8]

Reduced test case to

define ftrace_return_address(n) __builtin_return_address(n)

define CALLER_ADDR1 ftrace_return_address(1)

fn1() { fn2(CALLER_ADDR1); }