search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.26k stars 11.67k forks source link

Segmentation fault / null pointer exception #49526

Open 1d288b19-8adc-4084-80ae-9782093bedee opened 3 years ago

1d288b19-8adc-4084-80ae-9782093bedee commented 3 years ago
Bugzilla Link 50182
Version trunk
OS Linux
Attachments A minimal example program that causes the crash + Makefile that generates the three cases I described
CC @dwblaikie

Extended Description

When compiling one of the CFI-showcase programs (attached) with CFI instrumentation and recovery a nullpointer exception in llvm-symbolizer occurs.

The program is not clean by design - this is the point of a CFI showcase - but it runs fine without any instrumentation enabled.

It also runs fine with instrumentation enabled (-fsanitize=cfi-icall) - the violation is reported and the program aborts gracefully.

However, with instrumentation (-fsanitize=cfi-icall) AND recovery (-fsanitize-recover=all) enabled, the program does not resume execution after the CFI violation has been reported. A nullpointer exception causes a crash.

All other CFI showcase programs resume correctly, only this one case causes a crash.

Expected behavior: $ ./cfi_skip_entrypoint cfi_skip_entrypoint.c:35:15: runtime error: control flow integrity check for type 'int (int)' failed during indirect function call CFI ensures control flow only transfers to potentially valid destinations In not_entry_point: (0)

If you can read this, execution has resumed!

The program should report the violation and continue execution

Actual behavior: $ ./cfi_skip_entrypoint cfi_skip_entrypoint.c:35:15: runtime error: control flow integrity check for type 'int (int)' failed during indirect function call (/mnt/clang-cfi-showcase/cfi_skip_entrypoint+0x423c40): note: __libc_csu_init defined here SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior cfi_skip_entrypoint.c:35:15 in UndefinedBehaviorSanitizer:DEADLYSIGNAL ==23263==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000423c71 bp 0x000000086ce8 sp 0x7ffe258018d8 T23263) ==23263==The signal is caused by a READ memory access. ==23263==Hint: address points to the zero page.

​0 0x423c71 in __libc_csu_init (/mnt/clang-cfi-showcase/cfi_skip_entrypoint+0x423c71)

#​1 0x423c2f  (/mnt/clang-cfi-showcase/cfi_skip_entrypoint+0x423c2f)

UndefinedBehaviorSanitizer can not provide additional info. SUMMARY: UndefinedBehaviorSanitizer: SEGV (/mnt/clang-cfi-showcase/cfi_skip_entrypoint+0x423c71) in __libc_csu_init ==23263==ABORTING

The program reports the violation but cannot resume because of a nullpointer exception

This is what I got after debugging it in gdb:

Thread 2.1 "llvm-symbolizer" received signal SIGSEGV, Segmentation fault.

Run till exit from #​0 libc_start_main (main=0x423ad0

, argc=1, argv=0x7fffffffe758, init=0x423c30 <libc_csu_init>, fini=, rtld_fini=, stack_end=0x7fffffffe748) at ../csu/libc-start.c:268 cfi_skip_entrypoint.c:35:15: runtime error: control flow integrity check for type 'int (int)' failed during indirect function call

CFI violation is reported correctly <<<

[Attaching after Thread 0x7ffff78bdb80 (LWP 35794) fork to child process 35828] [New inferior 2 (process 35828)] [Detaching after fork from parent process 35794] [Inferior 1 (process 35794) detached] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". process 35828 is executing new program: /usr/lib/llvm-11/bin/llvm-symbolizer

Execution resumes, llvm-symbolizer is called <<<

[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Thread 2.1 "llvm-symbolizer" received signal SIGSEGV, Segmentation fault.

dwblaikie commented 3 years ago

Fair enough - thanks for giving it a go!

1d288b19-8adc-4084-80ae-9782093bedee commented 3 years ago

You might be able to reduce the test case a bit if you can figure out what's being passed to llvm-symbolizer and run just that - llvm-symbolizer shouldn't crash regardless (so that's at least one bug) - though it's possible it's being passed bogus/things from the sanitizer too (could be another bug).

I'm an undergrad student, so my skills are limited. But using strace I think I found out what is being passed to llvm-symbolizer:

... 14829 execve("/mnt/llvm-project/build/bin/llvm-symbolizer", ["/mnt/llvm-project/build/bin/llvm-symbolizer", "--inlines", "--default-arch=x86_64"], ["MAIL=/var/mail/marcel", "USER=marcel", "XDG_SEAT=seat0", "XDG_SESSION_TYPE=tty", "SHLVL=2", "HOME=/home/marcel", "MOTD_SHOWN=pam", "OLDPWD=/home/marcel", "HUSHLOGIN=FALSE", "LOGNAME=marcel", "JOURNALSTREAM=8:15158", "=/usr/bin/strace", "XDG_SESSION_CLASS=user", "TERM=xterm-kitty", "XDG_SESSION_ID=1", "WINDOWPATH=1", "PATH=/mnt/llvm-project/build/bin:/home/marcel/.local/share/texlive/2021/bin/ x86_64-linux:/home/marcel/.local/bin:usr/local/bin:/usr/sbin/:/usr/local/bin: /usr/bin:/bin:/usr/local/games:/usr/games", "INVOCATION_ID=3e7ecf722c854e1f85abd397636b6052", "XDG_RUNTIME_DIR=/run/user/1000", "DISPLAY=:0", "LANG=de_DE.UTF-8", "XAUTHORITY=/home/marcel/.Xauthority", "SHELL=/bin/zsh", "XDG_VTNR=1", "PWD=/mnt/clang-cfi-showcase", "SWM_STARTED=YES", "LD_PRELOAD=/usr/lib/spectrwm/libswmhack.so.0.0", "_SWM_WS=0", "_SWM_PID=12029", "KITTY_WINDOW_ID=1", "WINDOWID=71303182", "COLORTERM=truecolor", "LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64:", "MANPATH=/usr/local/man:/home/marcel/.local/share/texlive/2021/texmf-dist/ doc/man:", "INFOPATH=/home/marcel/.local/share/texlive/2021/texmf-dist/doc/info:", "EDITOR=vim", "VISUAL=vim", "MANPAGER=/bin/zsh -c \"vim -MRn -c \\"set buftype=nofile showtabline=0 ft=man ts=8 nomod nolist norelativenumber nonu noma\\" -c \\"normal L\\" -c \\"nmap q :qa\\"</dev/tty <(col -b)\"", "NNN_BMS=d:~/Dokumente;u:~/Dokumente/Uni/;D:~/Downloads;t:/tmp;m:~/Musik", "NNN_COLORS=6145", "NNN_PLUG=m:nmount", "LS_COLORS="]) = 0 ...

But I don't know if that is helpful. I wrote a minimal C program that calls this execve and compiled it with the same arguments and compiler as the other program and it does not crash.

The other thing that probably happens is that some addresses are fed to the standard input of llvm-symbolizer - it might be that reading/trying to process one of those addresses is what's causing the crash. So you'd probably need to discover/reproduce that standard input too.

I used a wrapper script from here:

and found this STDIN:

CODE "/mnt/clang-cfi-showcase/cfi_skip_entrypoint" 0x425691 CODE "/mnt/clang-cfi-showcase/cfi_skip_entrypoint" 0x42564f CODE "/mnt/clang-cfi-showcase/cfi_skip_entrypoint" 0x425691

to generate this STDOUT:

__libc_csu_init ??:0:0

?? ??:0:0

__libc_csu_init ??:0:0

But again, no crash when I run cat /tmp/logname-stdin | /mnt/llvm-project/build/bin/llvm-symbolizer --inlines --default-arch=x86_64

Perhaps the symbolizer is not the problem after all and gdb wrongly reported that there was the origin of the nullpointer exception? I'm not skilled enough to go on from here.

dwblaikie commented 3 years ago

You might be able to reduce the test case a bit if you can figure out what's being passed to llvm-symbolizer and run just that - llvm-symbolizer shouldn't crash regardless (so that's at least one bug) - though it's possible it's being passed bogus/things from the sanitizer too (could be another bug).

I'm an undergrad student, so my skills are limited. But using strace I think I found out what is being passed to llvm-symbolizer:

... 14829 execve("/mnt/llvm-project/build/bin/llvm-symbolizer", ["/mnt/llvm-project/build/bin/llvm-symbolizer", "--inlines", "--default-arch=x86_64"], ["MAIL=/var/mail/marcel", "USER=marcel", "XDG_SEAT=seat0", "XDG_SESSION_TYPE=tty", "SHLVL=2", "HOME=/home/marcel", "MOTD_SHOWN=pam", "OLDPWD=/home/marcel", "HUSHLOGIN=FALSE", "LOGNAME=marcel", "JOURNALSTREAM=8:15158", "=/usr/bin/strace", "XDG_SESSION_CLASS=user", "TERM=xterm-kitty", "XDG_SESSION_ID=1", "WINDOWPATH=1", "PATH=/mnt/llvm-project/build/bin:/home/marcel/.local/share/texlive/2021/bin/ x86_64-linux:/home/marcel/.local/bin:usr/local/bin:/usr/sbin/:/usr/local/bin: /usr/bin:/bin:/usr/local/games:/usr/games", "INVOCATION_ID=3e7ecf722c854e1f85abd397636b6052", "XDG_RUNTIME_DIR=/run/user/1000", "DISPLAY=:0", "LANG=de_DE.UTF-8", "XAUTHORITY=/home/marcel/.Xauthority", "SHELL=/bin/zsh", "XDG_VTNR=1", "PWD=/mnt/clang-cfi-showcase", "SWM_STARTED=YES", "LD_PRELOAD=/usr/lib/spectrwm/libswmhack.so.0.0", "_SWM_WS=0", "_SWM_PID=12029", "KITTY_WINDOW_ID=1", "WINDOWID=71303182", "COLORTERM=truecolor", "LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64:", "MANPATH=/usr/local/man:/home/marcel/.local/share/texlive/2021/texmf-dist/ doc/man:", "INFOPATH=/home/marcel/.local/share/texlive/2021/texmf-dist/doc/info:", "EDITOR=vim", "VISUAL=vim", "MANPAGER=/bin/zsh -c \"vim -MRn -c \\"set buftype=nofile showtabline=0 ft=man ts=8 nomod nolist norelativenumber nonu noma\\" -c \\"normal L\\" -c \\"nmap q :qa\\"</dev/tty <(col -b)\"", "NNN_BMS=d:~/Dokumente;u:~/Dokumente/Uni/;D:~/Downloads;t:/tmp;m:~/Musik", "NNN_COLORS=6145", "NNN_PLUG=m:nmount", "LS_COLORS="]) = 0 ...

But I don't know if that is helpful. I wrote a minimal C program that calls this execve and compiled it with the same arguments and compiler as the other program and it does not crash.

The other thing that probably happens is that some addresses are fed to the standard input of llvm-symbolizer - it might be that reading/trying to process one of those addresses is what's causing the crash. So you'd probably need to discover/reproduce that standard input too.

1d288b19-8adc-4084-80ae-9782093bedee commented 3 years ago

You might be able to reduce the test case a bit if you can figure out what's being passed to llvm-symbolizer and run just that - llvm-symbolizer shouldn't crash regardless (so that's at least one bug) - though it's possible it's being passed bogus/things from the sanitizer too (could be another bug).

I'm an undergrad student, so my skills are limited. But using strace I think I found out what is being passed to llvm-symbolizer:

... 14829 execve("/mnt/llvm-project/build/bin/llvm-symbolizer", ["/mnt/llvm-project/build/bin/llvm-symbolizer", "--inlines", "--default-arch=x86_64"], ["MAIL=/var/mail/marcel", "USER=marcel", "XDG_SEAT=seat0", "XDG_SESSION_TYPE=tty", "SHLVL=2", "HOME=/home/marcel", "MOTD_SHOWN=pam", "OLDPWD=/home/marcel", "HUSHLOGIN=FALSE", "LOGNAME=marcel", "JOURNALSTREAM=8:15158", "=/usr/bin/strace", "XDG_SESSION_CLASS=user", "TERM=xterm-kitty", "XDG_SESSION_ID=1", "WINDOWPATH=1", "PATH=/mnt/llvm-project/build/bin:/home/marcel/.local/share/texlive/2021/bin/x86_64-linux:/home/marcel/.local/bin:usr/local/bin:/usr/sbin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games", "INVOCATION_ID=3e7ecf722c854e1f85abd397636b6052", "XDG_RUNTIME_DIR=/run/user/1000", "DISPLAY=:0", "LANG=de_DE.UTF-8", "XAUTHORITY=/home/marcel/.Xauthority", "SHELL=/bin/zsh", "XDG_VTNR=1", "PWD=/mnt/clang-cfi-showcase", "SWM_STARTED=YES", "LD_PRELOAD=/usr/lib/spectrwm/libswmhack.so.0.0", "_SWM_WS=0", "_SWM_PID=12029", "KITTY_WINDOW_ID=1", "WINDOWID=71303182", "COLORTERM=truecolor", "LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64:", "MANPATH=/usr/local/man:/home/marcel/.local/share/texlive/2021/texmf-dist/doc/man:", "INFOPATH=/home/marcel/.local/share/texlive/2021/texmf-dist/doc/info:", "EDITOR=vim", "VISUAL=vim", "MANPAGER=/bin/zsh -c \"vim -MRn -c \\"set buftype=nofile showtabline=0 ft=man ts=8 nomod nolist norelativenumber nonu noma\\" -c \\"normal L\\" -c \\"nmap q :qa\\"</dev/tty <(col -b)\"", "NNN_BMS=d:~/Dokumente;u:~/Dokumente/Uni/;D:~/Downloads;t:/tmp;m:~/Musik", "NNN_COLORS=6145", "NNN_PLUG=m:nmount", "LS_COLORS="]) = 0 ...

But I don't know if that is helpful. I wrote a minimal C program that calls this execve and compiled it with the same arguments and compiler as the other program and it does not crash.

dwblaikie commented 3 years ago

You might be able to reduce the test case a bit if you can figure out what's being passed to llvm-symbolizer and run just that - llvm-symbolizer shouldn't crash regardless (so that's at least one bug) - though it's possible it's being passed bogus/things from the sanitizer too (could be another bug).