[Bug]: error: ran out of registers during register allocation

[kellermanrivero]

Description

Clang compiler for Arm (armv7) target fails to compile the following program:

./r27/bin/clang++ -I -fsanitize=address -c -o sample.o sample.cpp

sample.cpp:

#include <memory>
#include <boost/lockfree/queue.hpp>

class MyClass
{
public:
    MyClass()
    : m_queue(new boost::lockfree::queue<int>(1000))
    {
    }

private:
    std::unique_ptr<boost::lockfree::queue<int>> m_queue;
};

int main() {
    MyClass a;
    return 0;
}

The output of the compiler is:

➜  ~/Downloads/r27/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang++ -I `pwd` -fsanitize=address -c -o sample.o sample.cpp -v -Wno-deprecated-builtins
Android (11889484, +pgo, -bolt, +lto, -mlgo, based on r522817) clang version 18.0.1 (https://android.googlesource.com/toolchain/llvm-project d8003a456d14a3deb8054cdaa529ffbf02d9b262)
Target: armv7a-unknown-linux-android21
Thread model: posix
InstalledDir: /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin
 (in-process)
 "/Users/kellerms/pocs/android-ndk-compiler-issue/r27/bin/clang180++" -cc1 -triple armv7-unknown-linux-android21 -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name sample.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +soft-float-abi -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -sha2 -target-feature -aes -target-feature -fp16fml -target-abi aapcs-linux -mfloat-abi soft -debugger-tuning=gdb -fdebug-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -v -fcoverage-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -resource-dir /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18 -I /Users/kellerms/pocs/android-ndk-compiler-issue -internal-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/c++/v1 -internal-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/include -internal-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/local/include -internal-externc-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/arm-linux-androideabi -internal-externc-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/include -internal-externc-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include -Wno-deprecated-builtins -fdeprecated-macro -ferror-limit 19 -femulated-tls -fsanitize=address -fsanitize-system-ignorelist=/Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o sample.o -x c++ sample.cpp
clang -cc1 version 18.0.1 based upon LLVM 18.0.1 default target x86_64-apple-darwin23.5.0
ignoring nonexistent directory "/Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/local/include"
ignoring nonexistent directory "/Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/include"
#include "..." search starts here:
#include <...> search starts here:
 /Users/kellerms/pocs/android-ndk-compiler-issue
 /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/c++/v1
 /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/include
 /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/arm-linux-androideabi
 /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include
End of search list.
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation
6 errors generated.

This compiler error is reproducible since R24. At least during my tests I found r23c seems to be able to compile this piece of code without major issues:

➜  ~/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang++ -I `pwd` -fsanitize=address -c -o sample.o sample.cpp -v -Wno-deprecated-builtins
Android (8481493, based on r416183c2) clang version 12.0.9 (https://android.googlesource.com/toolchain/llvm-project c935d99d7cf2016289302412d708641d52d2f7ee)
Target: armv7a-unknown-linux-android21
Thread model: posix
InstalledDir: /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin
Found candidate GCC installation: /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x
Selected GCC installation: /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x
 (in-process)
 "/Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++" -cc1 -triple armv7-unknown-linux-android21 -emit-obj -mrelax-all --mrelax-relocations -mnoexecstack -disable-free -disable-llvm-verifier -discard-value-names -main-file-name sample.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu generic -target-feature +soft-float-abi -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -crypto -target-feature -fp16fml -target-abi aapcs-linux -mfloat-abi soft -fallow-half-arguments-and-returns -fno-split-dwarf-inlining -debugger-tuning=gdb -target-linker-version 711 -v -resource-dir /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/12.0.9 -I /Users/kellerms/pocs/android-ndk-compiler-issue -internal-isystem /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/include/c++/v1 -internal-isystem /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/local/include -internal-isystem /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/12.0.9/include -internal-externc-isystem /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/include/arm-linux-androideabi -internal-externc-isystem /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/include -internal-externc-isystem /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/include -Wno-deprecated-builtins -fdeprecated-macro -fdebug-compilation-dir /Users/kellerms/pocs/android-ndk-compiler-issue -ferror-limit 19 -fsanitize=address -fsanitize-system-blacklist=/Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/12.0.9/share/asan_blacklist.txt -fsanitize-address-use-after-scope -fno-assume-sane-operator-new -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -o sample.o -x c++ sample.cpp
warning: unknown warning option '-Wno-deprecated-builtins'; did you mean '-Wno-deprecated-volatile'? [-Wunknown-warning-option]
clang -cc1 version 12.0.9 based upon LLVM 12.0.9git default target x86_64-apple-darwin23.5.0
ignoring nonexistent directory "/Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/include"
#include "..." search starts here:
#include <...> search starts here:
 /Users/kellerms/pocs/android-ndk-compiler-issue
 /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/include/c++/v1
 /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/local/include
 /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/12.0.9/include
 /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/include/arm-linux-androideabi
 /Users/kellerms/Downloads/r23c/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/include
End of search list.
1 warning generated.

I'm unable to reproduce this bug with upstream LLVM so I'm not sure if this is specific of Android NDK.

Upstream bug

No response

Commit to cherry-pick

No response

Affected versions

r26, r27

Canary version

No response

Host OS

Linux, Mac

Host OS version

Amazon Linux 2, MacOS 14.5 (Sonoma)

Affected ABIs

armeabi-v7a

[kellermanrivero]

I have tested with stock LLVM/Clang in the following way:

➜  /opt/homebrew/opt/llvm/bin/clang++ -target armv7a-unknown-linux-android21 -I `<path_to_boost_include_dir>` -c -o sample.o sample.cpp -v -Wno-deprecated-builtins -fsanitize=address --sysroot ~/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot
Homebrew clang version 18.1.7
Target: armv7a-unknown-linux-android21
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
 (in-process)
 "/opt/homebrew/Cellar/llvm/18.1.7/bin/clang-18" -cc1 -triple armv7-unknown-linux-android21 -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name sample.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +soft-float-abi -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -sha2 -target-feature -aes -target-feature -fp16fml -target-abi aapcs-linux -mfloat-abi soft -debugger-tuning=gdb -fdebug-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -target-linker-version 1053.12 -v -fcoverage-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -resource-dir /opt/homebrew/Cellar/llvm/18.1.7/lib/clang/18 -I /Users/kellerms/pocs/android-ndk-compiler-issue -isysroot /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -internal-isystem /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1 -internal-isystem /opt/homebrew/Cellar/llvm/18.1.7/lib/clang/18/include -internal-isystem /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/local/include -internal-externc-isystem /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/arm-linux-androideabi -internal-externc-isystem /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/include -internal-externc-isystem /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include -Wno-deprecated-builtins -fdeprecated-macro -ferror-limit 19 -femulated-tls -fsanitize=address -fsanitize-system-ignorelist=/opt/homebrew/Cellar/llvm/18.1.7/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fno-signed-char -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o sample.o -x c++ sample.cpp
clang -cc1 version 18.1.7 based upon LLVM 18.1.7 default target arm64-apple-darwin23.5.0
ignoring nonexistent directory "/Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/local/include"
ignoring nonexistent directory "/Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/include"
#include "..." search starts here:
#include <...> search starts here:
 /Users/kellerms/pocs/android-ndk-compiler-issue
 /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1
 /opt/homebrew/Cellar/llvm/18.1.7/lib/clang/18/include
 /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/arm-linux-androideabi
 /Users/kellerms/Downloads/r26d/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include
End of search list.

And it's able to compile the sample program without issue. This makes me thing is something with Android flavor of Clang.

[DanAlbert]

To clarify: there is no android flavor of Clang. We built it, but it's the upstream source. The reason for the behavior difference is that it's a different version of Clang, which does mean that this bug might be fixed in r28, which has a newer Clang than either r26 or r27.

It's not likely that we'll have a backport for r26 at this point, as it doesn't sound like this is a regression from r25 (though we'll need to confirm that). r27 is a possibility though.

[pirama-arumuga-nainar]

Can you add --save-temps and share the sample.ii (with r27, like danalbert mentioned)? To make sure we are looking at the same failure as yours.

[kellermanrivero]

Hello, output running with --save-temps:

➜  android-ndk-compiler-issue ./r27/bin/armv7a-linux-androideabi21-clang++ -I `pwd` -c -o sample.o sample.cpp -v -Wno-deprecated-builtins -fsanitize=address --save-temps
Android (11889484, +pgo, -bolt, +lto, -mlgo, based on r522817) clang version 18.0.1 (https://android.googlesource.com/toolchain/llvm-project d8003a456d14a3deb8054cdaa529ffbf02d9b262)
Target: armv7a-unknown-linux-android21
Thread model: posix
InstalledDir: /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin
 "/Users/kellerms/pocs/android-ndk-compiler-issue/r27/bin/clang180++" -cc1 -triple armv7-unknown-linux-android21 -E -save-temps=cwd -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name sample.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +soft-float-abi -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -sha2 -target-feature -aes -target-feature -fp16fml -target-abi aapcs-linux -mfloat-abi soft -debugger-tuning=gdb -fdebug-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -v -fcoverage-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -resource-dir /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18 -I /Users/kellerms/pocs/android-ndk-compiler-issue -internal-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/c++/v1 -internal-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/include -internal-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/local/include -internal-externc-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/arm-linux-androideabi -internal-externc-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/include -internal-externc-isystem /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include -Wno-deprecated-builtins -fdeprecated-macro -ferror-limit 19 -femulated-tls -fsanitize=address -fsanitize-system-ignorelist=/Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o sample.ii -x c++ sample.cpp
clang -cc1 version 18.0.1 based upon LLVM 18.0.1 default target x86_64-apple-darwin23.5.0
ignoring nonexistent directory "/Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/local/include"
ignoring nonexistent directory "/Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/include"
#include "..." search starts here:
#include <...> search starts here:
 /Users/kellerms/pocs/android-ndk-compiler-issue
 /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/c++/v1
 /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/include
 /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include/arm-linux-androideabi
 /Users/kellerms/pocs/android-ndk-compiler-issue/./r27/bin/../sysroot/usr/include
End of search list.
 "/Users/kellerms/pocs/android-ndk-compiler-issue/r27/bin/clang180++" -cc1 -triple armv7-unknown-linux-android21 -emit-llvm-bc -emit-llvm-uselists -save-temps=cwd -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name sample.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +soft-float-abi -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -sha2 -target-feature -aes -target-feature -fp16fml -target-abi aapcs-linux -mfloat-abi soft -debugger-tuning=gdb -fdebug-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -v -fcoverage-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -resource-dir /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18 -Wno-deprecated-builtins -fdeprecated-macro -ferror-limit 19 -femulated-tls -fsanitize=address -fsanitize-system-ignorelist=/Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -disable-llvm-passes -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o sample.bc -x c++-cpp-output sample.ii
clang -cc1 version 18.0.1 based upon LLVM 18.0.1 default target x86_64-apple-darwin23.5.0
#include "..." search starts here:
End of search list.
 "/Users/kellerms/pocs/android-ndk-compiler-issue/r27/bin/clang180++" -cc1 -triple armv7-unknown-linux-android21 -S -save-temps=cwd -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name sample.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +soft-float-abi -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -sha2 -target-feature -aes -target-feature -fp16fml -target-abi aapcs-linux -mfloat-abi soft -debugger-tuning=gdb -fdebug-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -v -fcoverage-compilation-dir=/Users/kellerms/pocs/android-ndk-compiler-issue -resource-dir /Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18 -Wno-deprecated-builtins -ferror-limit 19 -femulated-tls -fsanitize=address -fsanitize-system-ignorelist=/Users/kellerms/pocs/android-ndk-compiler-issue/r27/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fno-signed-char -fgnuc-version=4.2.1 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o sample.s -x ir sample.bc
clang -cc1 version 18.0.1 based upon LLVM 18.0.1 default target x86_64-apple-darwin23.5.0
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation
error: ran out of registers during register allocation

Here is a zip file with temp files: temps.zip

[kellermanrivero]

To clarify: there is no android flavor of Clang. We built it, but it's the upstream source. The reason for the behavior difference is that it's a different version of Clang, which does mean that this bug might be fixed in r28, which has a newer Clang than either r26 or r27.

It's not likely that we'll have a backport for r26 at this point, as it doesn't sound like this is a regression from r25 (though we'll need to confirm that). r27 is a possibility though.

Thanks for the clarification. Our stack was previously being built with R18 (I know... a little bit old) and we only found this issue once we tried to move to R26. I have tested R22 & R23 and both seems to compile fine this piece of code. So definitely something introduced in R24.

I dive deep into why boost lockfree queue triggers this behavior and it seems to be because the heavy use of templates to deduce compile/runtime size of the queue type. I tried to isolate an smaller program without use of boost library without success. Adding this context just in case is somehow helpful (I hope so).

[DanAlbert]

We'll see what we can do, but we do give priority to regressions from the previous release over bugs that have been broken for a long time (if they've been broken a long time and this is the first report, they probably aren't as wide spread). We're almost entirely dependent on upstream actually providing the fixes, but since it's working with LLVM 18 (and I think r27 is ever so slightly behind LLVM 18?), the odds are pretty good that there's already a fix we can cherry-pick.

[appujee]

clang-18.0.0(r510928) doesn't crash but 18.0.1(r522817) does.

[appujee]

Simpler command line to repro with the provided sample.ii file.

$CC -cc1 -triple armv7-unknown-linux-android21 -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name sample.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +soft-float-abi -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -sha2 -target-feature -aes -target-feature -fp16fml -target-abi aapcs-linux -mfloat-abi soft -fdeprecated-macro -ferror-limit 19 -femulated-tls -fsanitize=address  -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-assume-sane-operator-new -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o sample.o -x c++ sample.ii

[appujee]

Using -mllvm -asan-use-stack-safety=1 fixes the error. I feel the problem is still latent in llvm but after https://github.com/llvm/llvm-project/pull/77210 the error went away.

Either cherry-picking this patch to ndk or maybe adding this flag as default cc flags should be fine.

[appujee]

Yeah so even with trunk compiler and with -mllvm -asan-use-stack-safety=0 we get this error. ~The error only occurs for Android targets though~. https://godbolt.org/z/5ocv7Wsa9

[DanAlbert]

I hadn't read closely enough before (evidently I really needed that vacation) and missed that this was ASan related. @kellermanrivero, have you tried HWASan? ASan isn't actually supported for Android by upstream any more because there are so few reasons why it should be used over HWASan. See https://developer.android.com/ndk/guides/memory-debug for an overview of the tools available. ASan says "only as a last resort". HWASan was historically a pain to use, but as of U (which I know is extremely new) it's easier to use than ASan is.

Clang shouldn't crash in any case and it'd be good to pick up the fix for this, but if you've got a better alternative you may not actually care :)

[kellermanrivero]

@appujee great job! thanks for the dive deep here. It's extremely valuable. I feel I have somehow contributed to the project by reporting this error.

@DanAlbert my team is moving away from a really old NDK r18 into latest one so I guess maybe ASAN was the recommended option for R18. I will try out HWASan and report the results back.

Thanks both for the hard work here!

[DanAlbert]

@DanAlbert my team is moving away from a really old NDK r18 into latest one so I guess maybe ASAN was the recommended option for R18. I will try out HWASan and report the results back.

r18 is old enough that I don't remember whether hwasan would work there or not, but I suspect you're right. Even if it did, I think you probably need an NDK that is aware of android U (which I think means r26+) to use the easy mode?

[DanAlbert]

Tentatively adding this for r27b so we remember to check, but due to the lack of upstream support for ASan, there may never be a fix for us to pick up.

[kellermanrivero]

@DanAlbert I just realized that this issue specifically happens on Armv7 target so HWASan won't work since it only supports 64-bit architecture.

[DanAlbert]

Maybe. If the bug is present but asymptomatic on arm64, hwasan should still be able to catch it. If it's in arm32-specific code though, yeah, hwasan won't help you :(

[appujee]

The reported bug in llvm-upstream has a reduced testcase, maybe that can give you some insight on a workaround in the mean time? https://github.com/llvm/llvm-project/issues/98222