CFI tests not working on ARM (buildbot failure)

[smithp35]

Bugzilla Link	38690
Version	7.0
OS	Linux

Extended Description

We have recently introduced a 2-stage clang LLD builder and I think it has uncovered some latent problems with the CFI tests.

Builder URL: http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/

I think that there are 3 separate problems:

The first problem is that the cmake logic for tests that require LLD (for thin-lto) doesn't enable the tests from a clean build. This isn't ARM specific. The problem is that COMPILER_RT_HAS_LLD depends on LLVM_TOOL_LLD_BUILD which is set after the code in compiler-rt/CMakeLists.txt is run. I found I needed to modify the compiler-rt/cfi/CMakeLists.txt file in order to get the LLD dependent tests to run.

The second problem is that when the tests run on ARM there are assertion failures in the tests when assertions are enabled. I think that this is due to an assertion about the address of _cfi_check that doesn't hold on Thumb. Specifically: void ShadowBuilder::Add(uptr begin, uptr end, uptr cfi_check) { assert((cfi_check & (kShadowAlign - 1)) == 0);

The cfi_check function is always compiled to be Thumb and that means it will have bit 0 set to 1 (to mark it as Thumb) so it will always fail that assertion.

The third problem is that the cross_dso tests fail when assertions are not enabled. I don't have any more information other than I ran one of the test binaries manually and it segfaulted.

At this stage I don't think it is LLD that is to blame, if I use gold I get more failures.

Unfortunately this will probably need a native Arm machine to run the test, qemu-arm can sometimes be used but sanitizers can often cause the user-mode emulator problems. I'll try and provide an example that will be buildable on an x86 machine but it may take some time.

The cmake files I used with a monorepo checkout: -DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt" \ -DCMAKE_BUILD_TYPE=Release \ -DLLVM_ENABLE_ASSERTIONS=true\ -DCMAKE_C_FLAGS='-mcpu=cortex-a57'\ -DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'\ -DLLVM_TARGETS_TO_BUILD='ARM;AArch64'\ -DLLVM_ENABLE_LLD=true

The compiler used was clang6.0 downloaded from http://releases.llvm.org/download.html

I found I needed to touch the compiler-rt/cfi/CMakeLists.txt in order to provoke it to run the cfi tests with LLD.

[DavidSpickett]

The following currently passes all tests:

cmake ../llvm-project/llvm -G Ninja -DLLVM_ENABLE_PROJECTS="clang;lld" -DLLVM_ENABLE_RUNTIMES=compiler-rt \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_ASSERTIONS=true \
  -DCMAKE_C_FLAGS='-mcpu=cortex-a57' \
  -DCMAKE_CXX_FLAGS='-mcpu=cortex-a57' \
  -DLLVM_ENABLE_LLD=true \
  -DLLVM_LIT_ARGS=-v \
  -DLLVM_TARGETS_TO_BUILD=ARM && ninja

So we can re-enable sanitizers on this bot https://github.com/llvm/llvm-zorg/blob/e3a5b28f7fc6af8820441f0f11c6cded87f18acf/buildbot/osuosl/master/config/builders.py#L431.

I spoke too soon, all the tests are marked unsupported. Trying to figure out how to get them to acutually run.

[DavidSpickett]

These tests are not being run on our AArch64 lld bot either: https://lab.llvm.org/buildbot/#/builders/185/builds/3349

[DavidSpickett]

If you don't use the runtimes build some will run.

Failed Tests (12):
  cfi-devirt-lld-armhf :: cross-dso/simple-fail.cpp
  cfi-devirt-lld-armhf :: cross-dso/simple-pass.cpp
  cfi-devirt-lld-armhf :: cross-dso/stats.cpp
  cfi-devirt-lld-thinlto-armhf :: cross-dso/simple-fail.cpp
  cfi-devirt-lld-thinlto-armhf :: cross-dso/simple-pass.cpp
  cfi-devirt-lld-thinlto-armhf :: cross-dso/stats.cpp
  cfi-standalone-lld-armhf :: cross-dso/simple-fail.cpp
  cfi-standalone-lld-armhf :: cross-dso/simple-pass.cpp
  cfi-standalone-lld-armhf :: cross-dso/stats.cpp
  cfi-standalone-lld-thinlto-armhf :: cross-dso/simple-fail.cpp
  cfi-standalone-lld-thinlto-armhf :: cross-dso/simple-pass.cpp
  cfi-standalone-lld-thinlto-armhf :: cross-dso/stats.cpp

Testing Time: 5.27s
  Unsupported      : 162
  Passed           :  74
  Expectedly Failed:   8
  Failed           :  12

So there are at least 2 issues:

• Failures on Arm.
• Runtimes build not enabling as many tests.

And probably still the issue of building for Thumb.