systemz: "error in backend: Unsupported stack frame traversal count"

[Quuxplusone]

Bugzilla Link	PR41424
Status	NEW
Importance	P enhancement
Reported by	Arnd Bergmann (arnd@linaro.org)
Reported on	2019-04-08 05:49:52 -0700
Last modified on	2019-04-17 09:08:47 -0700
Version	8.0
Hardware	Other Linux
CC	htmldeveloper@gmail.com, llvm-bugs@lists.llvm.org, ndesaulniers@google.com, uweigand@de.ibm.com
Fixed by commit(s)
Attachments	trace_sched_wakeup-ff17db.c.gz (911787 bytes, application/gzip) trace_sched_wakeup-ff17db.sh (5958 bytes, application/x-shellscript)
Blocks	PR4068
Blocked by
See also

Created attachment 21745
preprocessed kernel/trace/trace_sched_wakeup.c

I get a build failure for various file when trying to compile the linux kernel
for s390 (systemz):

clang-8" "-cc1" "-triple" "s390x-unknown-linux-gnu" "-S" "-disable-free" "-
disable-llvm-verifier" "-discard-value-names" "-main-file-name"
"trace_sched_wakeup.c" "-mrelocation-model" "static" "-mthread-model" "posix" "-
relaxed-aliasing" "-fmath-errno" "-masm-verbose" "-no-integrated-as" "-
mconstructor-aliases" "-fuse-init-array" "-target-cpu" "z196" "-mbackchain" "-
dwarf-column-info" "-debug-info-kind=limited" "-dwarf-version=4" "-debugger-
tuning=gdb" "-momit-leaf-frame-pointer" "-coverage-notes-file" "/git/arm-
soc/y2038/obj-s390/kernel/trace/trace_sched_wakeup.gcno" "-nostdsysteminc" "-
nobuiltininc" "-sys-header-deps" "-D" "__KERNEL__" "-D"
"CONFIG_AS_CFI_VAL_OFFSET=1" "-D" "CC_USING_FENTRY" "-D"
"KBUILD_BASENAME=\"trace_sched_wakeup\"" "-D"
"KBUILD_MODNAME=\"trace_sched_wakeup\"" "-O2" "-Wall" "-Wundef" "-Werror=strict-
prototypes" "-Wno-trigraphs" "-Werror=implicit-function-declaration" "-
Werror=implicit-int" "-Wno-format-security" "-Wno-sign-compare" "-Wno-format-
invalid-specifier" "-Wno-gnu" "-Wno-address-of-packed-member" "-Wno-
tautological-compare" "-Wno-unused-const-variable" "-Wdeclaration-after-
statement" "-Wvla" "-Wno-pointer-sign" "-Werror=date-time" "-
Werror=incompatible-pointer-types" "-Wno-initializer-overrides" "-Wno-unused-
value" "-Wno-format" "-Wno-sign-compare" "-Wno-format-zero-length" "-std=gnu89"
"-fno-dwarf-directory-asm" "-ferror-limit" "19" "-fmessage-length" "132" "-
fwrapv" "-fno-signed-char" "-fwchar-type=short" "-fno-signed-wchar" "-fobjc-
runtime=gcc" "-fno-common" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-
vectorize-loops" "-vectorize-slp" "-x" "c" "trace_sched_wakeup-ff17db.c"

fatal error: error in backend: Unsupported stack frame traversal count

I did not try reducing the test case, but could do so if it's not immediately
clear from the output where the problem is.

[Quuxplusone]

Attached trace_sched_wakeup-ff17db.c.gz (911787 bytes, application/gzip): preprocessed kernel/trace/trace_sched_wakeup.c

[Quuxplusone]

Attached trace_sched_wakeup-ff17db.sh (5958 bytes, application/x-shellscript): reproducer sript for sched_wakeup.c

[Quuxplusone]

Reduced test case to

#define ftrace_return_address(n) __builtin_return_address(n)
#define CALLER_ADDR1 ftrace_return_address(1)
fn1() { fn2(CALLER_ADDR1); }

[Quuxplusone]

Just like GCC, LLVM will not allow __builtin_return_address(N) for any N > 0 in the absence of a backchain (because there is no way to retrieve the return address in higher frames -- except by reading DWARF).

GCC does support builtin_return_address(N) for N > 0 if you also build with backchain (-mbackchain). LLVM doesn't do that even that. This is because originally LLVM didn't support -mbackchain at all. While it now does, we didn't change the builtin_return_address implementation.

If this is important, I think we could add the same support as is in GCC today. As in GCC, it would only work reliably if the whole code base is built with -mbackchain.

I understand that even the kernel doesn't really want the stack size overhead involved with -mbackchain, which is why a while back we implemented a special -mkernel-backchain feature in GCC just for the kernel. This isn't implemented in LLVM at all at the moment.

Why is it a requirement to build the kernel with LLVM? None of the distros do as far as I know ...

[Quuxplusone]

(In reply to Ulrich Weigand from comment #3)

> If this is important, I think we could add the same support as is in GCC
> today.  As in GCC, it would only work reliably if the whole code base is
> built with -mbackchain.
>
> I understand that even the kernel doesn't really want the stack size
> overhead involved with -mbackchain, which is why a while back we implemented
> a special -mkernel-backchain feature in GCC just for the kernel.  This isn't
> implemented in LLVM at all at the moment.

Ok, I see.

> Why is it a requirement to build the kernel with LLVM?  None of the distros
> do as far as I know ...

Correct: to my knowledge, there is currently nobody trying to use such a kernel
in practice. My motivation was purely for build testing, and making sure that
the clang binaries I built for creating ARM kernels work across other
architectures as well.

[Quuxplusone]

I submitted a kernel patch to avoid the use __builtin_return_address() with
clang, see

https://lore.kernel.org/lkml/20190408212648.2407234-10-arnd@arndb.de/

Including that one in the kernel should be sufficient. However, it would be
helpful to have a more specific error message here (i.e. including the source
code location), and possibly turn it into a warning, to match the gcc behavior
when backchains are disabled. On the reduced test case, gcc only warns

test.c:3:9: warning: unsupported argument to '__builtin_return_address'
 fn1() { fn2(CALLER_ADDR1); }
         ^~~~~~~~~~~~~~~~~

[Quuxplusone]

Why is it a requirement to build the kernel with LLVM? None of the distros do as far as I know ...

Kind of a non-starter if there's missing features the kernel depends on.

[Quuxplusone]

In general I'm happy to add features to LLVM where it makes sense to support building the kernel. In this particular case it doesn't seem necessary.

I've just talked to Martin and he confirms that the kernel (also on SystemZ) is moving to using ORC for unwinding, which implies that neither -mkernel-backchain nor __builtin_return_address(n) with n>0 will be used any more.

The only remaining issue is that it would indeed be nicer if we got a front-end warning instead of a back-end internal error. Unfortunately that's a deficiency in clang at the moment (__builtin_return_address is handled by platform-independent code that is not aware of platform-specific limitations). I'll see what I can do about that.