Assertion `!isa<PointerType>(v1->getType())&& " Generating Pointer and not being handled"' failed

[andersk]

I decided to play with SoftBound + CETS to see if it could have been helpful in diagnosing a GCC bug, and I happened across some code that causes an assertion failure in SoftBoundCETSPass::gatherBaseBoundPass1. Here’s a reduced test case.

$ cat foo.c
struct node { struct node *next, *prev; } *n;
struct head { struct node *first, *last; } h;
void fn() {
  h.last = n->prev;
  if (n->prev != (void *)&h)
    h.first = n->next;
  else
    n->prev->next = n->next;
  n->next = h.first;
}
int main() { fn(); }

$ clang -fsoftboundcets -O2 -c foo.c
clang: SoftBoundCETS.cpp:4608: void SoftBoundCETSPass::gatherBaseBoundPass1(llvm::Function *): Assertion `!isa<PointerType>(v1->getType())&& " Generating Pointer and not being handled"' failed.
0  clang           0x0000000002275c85 llvm::sys::PrintStackTrace(_IO_FILE*) + 37
1  clang           0x00000000022760d3
2  libpthread.so.0 0x00007f37f10ee320
3  libc.so.6       0x00007f37f00f6117 gsignal + 55
4  libc.so.6       0x00007f37f00f7808 abort + 328
5  libc.so.6       0x00007f37f00eefc6
6  libc.so.6       0x00007f37f00ef072
7  clang           0x0000000001469e91
8  clang           0x000000000146bf25 SoftBoundCETSPass::runOnModule(llvm::Module&) + 837
9  clang           0x00000000022091c7 llvm::legacy::PassManagerImpl::run(llvm::Module&) + 999
10 clang           0x000000000084b260 clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::Module*, clang::BackendAction, llvm::raw_ostream*) + 6096
11 clang           0x0000000000848e7e
12 clang           0x00000000009cb113 clang::ParseAST(clang::Sema&, bool, bool) + 515
13 clang           0x0000000000847e3f clang::CodeGenAction::ExecuteAction() + 143
14 clang           0x00000000006b3d20 clang::FrontendAction::Execute() + 112
15 clang           0x000000000069107d clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 925
16 clang           0x0000000000677cd0 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 2896
17 clang           0x000000000066f86f cc1_main(char const**, char const**, char const*, void*) + 687
18 clang           0x0000000000675da8 main + 10008
19 libc.so.6       0x00007f37f00e0ec5 __libc_start_main + 245
20 clang           0x000000000066f4ed
Stack dump:
0.  Program arguments: /home/anders/wd/softboundcets-34/softboundcets-llvm-clang34/Release+Asserts/bin/clang -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -disable-free -main-file-name foo.c -mrelocation-model static -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -target-linker-version 2.24.51.20140807 -momit-leaf-frame-pointer -coverage-file /tmp/foo.o -resource-dir /home/anders/wd/softboundcets-34/softboundcets-llvm-clang34/Release+Asserts/bin/../lib/clang/3.4 -internal-isystem /usr/local/include -internal-isystem /home/anders/wd/softboundcets-34/softboundcets-llvm-clang34/Release+Asserts/bin/../lib/clang/3.4/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O2 -fdebug-compilation-dir /tmp -ferror-limit 19 -fmessage-length 186 -fsoftboundcets -mstackrealign -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o foo.o -x c foo.c 
1.  <eof> parser at end of file
2.  Per-module optimization passes
3.  Running pass ' SoftBoundCETSPass' on module 'foo.c'.
clang: error: unable to execute command: Aborted (core dumped)
clang: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 3.4 (branches/release_34)
Target: x86_64-unknown-linux-gnu
Thread model: posix
clang: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/foo-64e14c.c
clang: note: diagnostic msg: /tmp/foo-64e14c.sh
clang: note: diagnostic msg: 

********************

$ diff -u foo.c /tmp/foo-64e14c.c
--- foo.c   2014-08-09 19:18:53.080250931 -0400
+++ /tmp/foo-64e14c.c   2014-08-09 19:21:49.660255453 -0400
@@ -1,3 +1,5 @@
+# 1 "<built-in>" 1
+# 1 "foo.c" 1
 struct node { struct node *next, *prev; } *n;
 struct head { struct node *first, *last; } h;
 void fn() {

$ cat /tmp/foo-64e14c.sh
 "/home/anders/wd/softboundcets-34/softboundcets-llvm-clang34/Release+Asserts/bin/clang" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -disable-free -main-file-name foo.c -mrelocation-model static -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -target-linker-version 2.24.51.20140807 -momit-leaf-frame-pointer -O2 -ferror-limit 19 -fmessage-length 186 -fsoftboundcets -mstackrealign -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -x c foo-64e14c.c

[santoshn]

Thanks for reporting the issue. The problem appears because LLVM is vectorizing the load and store of pointers and subsequently using extractvalue to access the vector elements. Currently, softboundcets does not allow extractvalue/insertvalue to return pointer values. I will update with a patch to address this issue.

[santoshn]

Fixed the compilation error with the latest commit. Now SoftboundCETS reports a memory safety violation with the given test case.