search

google / sanitizers

AddressSanitizer, ThreadSanitizer, MemorySanitizer
Other
11.45k stars 1.03k forks source link

msan: False positive with libaio? #688

Closed sitsofe closed 8 years ago

sitsofe commented 8 years ago

When trying to use a program that uses libaio MemorySanitzer regularly says uninitialized values are stored just after the io_getevents call:

==20626==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x49842e in finish_io /tmp/aio-stress.c:450:32
    #1 0x4b31d9 in io_oper_wait /tmp/aio-stress.c:548:2
    #2 0x4ac99e in worker /tmp/aio-stress.c:1145:2
    #3 0x4bc18b in main /tmp/aio-stress.c:1499:11
    #4 0x7efc54837f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)
    #5 0x419f6e in _start (/tmp/a.out+0x419f6e)

  Uninitialized value was stored to memory at
    #0 0x4982bb in finish_io /tmp/aio-stress.c:449
    #1 0x4b31d9 in io_oper_wait /tmp/aio-stress.c:548:2
    #2 0x4ac99e in worker /tmp/aio-stress.c:1145:2
    #3 0x4bc18b in main /tmp/aio-stress.c:1499:11
    #4 0x7efc54837f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)

  Uninitialized value was stored to memory at
    #0 0x4b30dd in io_oper_wait /tmp/aio-stress.c:545:18
    #1 0x4ac99e in worker /tmp/aio-stress.c:1145:2
    #2 0x4bc18b in main /tmp/aio-stress.c:1499:11
    #3 0x7efc54837f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)

  Uninitialized value was created by an allocation of 'event' in the stack frame of function 'io_oper_wait'
    #0 0x4b28f0 in io_oper_wait /tmp/aio-stress.c:525

SUMMARY: MemorySanitizer: use-of-uninitialized-value /tmp/aio-stress.c:450:32 in finish_io
Exiting

I see that the sanitizers know about the syscall (https://github.com/llvm-mirror/compiler-rt/blob/release_38/lib/sanitizer_common/sanitizer_common_syscalls.inc#L1295 ) but even if I compile libaio by hand with memory sanitizer and use LD_LIBRARY_PATH to ensure my hand compiled version is used I still get errors.

Steps to reproduce:

  1. Download the xfstests version of aio-stress.c (http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=blob;f=ltp/aio-stress.c;hb=HEAD ).
  2. Compile aio-stress by doing clang-3.8 -g -O0 -fsanitize=memory -fsanitize-memory-track-origins=2 -lpthread -laio -o aio-stress aio-stress.c
  3. Run ./aio-stress -m -s 2MB

Expected results: Test to run without error?

Actual result:

$ ./aio-stress -m -s 2MB
file size 1024MB, record size 64KB, depth 64, ios per iteration 8
max io_submit 8, buffer alignment set to 4KB
threads 1 files 1 contexts 1 context offset 2MB verification off
Running single thread version 
==20656==WARNING: MemorySanitizer: use-of-uninitialized-value
[...]

How reproducible is the problem? The problem can be reproduced every time.

Version information: Clang 3.8 Ubuntu 14.04 x86_64

eugenis commented 8 years ago

The problem is that libaio uses inline asm for those syscalls, and MSan does not intercept it. It can be fixed in libaio code with something like this:

include <sanitizer/linux_syscall_hooks.h>

define PRE(name, ...) sanitizer_syscallpre##name(__VA_ARGS)

define POST(name, res, ...) **sanitizer_syscallpost##name(res,

__VA_ARGS**)

and then add PRE and POST to io_syscall1 .. io_syscall5.

On Sat, Jun 11, 2016 at 1:03 AM, Sitsofe Wheeler notifications@github.com wrote:

When trying to use a program that uses libaio MemorySanitzer regularly says uninitialized values are stored just after the io_getevents call:

==20626==WARNING: MemorySanitizer: use-of-uninitialized-value

0 0x49842e in finish_io /tmp/aio-stress.c:450:32

#1 0x4b31d9 in io_oper_wait /tmp/aio-stress.c:548:2
#2 0x4ac99e in worker /tmp/aio-stress.c:1145:2
#3 0x4bc18b in main /tmp/aio-stress.c:1499:11
#4 0x7efc54837f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)
#5 0x419f6e in _start (/tmp/a.out+0x419f6e)

Uninitialized value was stored to memory at

0 0x4982bb in finish_io /tmp/aio-stress.c:449

#1 0x4b31d9 in io_oper_wait /tmp/aio-stress.c:548:2
#2 0x4ac99e in worker /tmp/aio-stress.c:1145:2
#3 0x4bc18b in main /tmp/aio-stress.c:1499:11
#4 0x7efc54837f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)

Uninitialized value was stored to memory at

0 0x4b30dd in io_oper_wait /tmp/aio-stress.c:545:18

#1 0x4ac99e in worker /tmp/aio-stress.c:1145:2
#2 0x4bc18b in main /tmp/aio-stress.c:1499:11
#3 0x7efc54837f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)

Uninitialized value was created by an allocation of 'event' in the stack frame of function 'io_oper_wait'

0 0x4b28f0 in io_oper_wait /tmp/aio-stress.c:525

SUMMARY: MemorySanitizer: use-of-uninitialized-value /tmp/aio-stress.c:450:32 in finish_io Exiting

I see that the sanitizers know about the syscall ( https://github.com/llvm-mirror/compiler-rt/blob/release_38/lib/sanitizer_common/sanitizer_common_syscalls.inc#L1295 ) but even if I compile libaio by hand with memory sanitizer and use LD_LIBRARY_PATH to ensure my hand compiled version is used I still get errors.

Steps to reproduce:

  1. Download the xfstests version of aio-stress.c ( http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=blob;f=ltp/aio-stress.c;hb=HEAD ).
  2. Compile aio-stress by doing clang-3.8 -g -O0 -fsanitize=memory -fsanitize-memory-track-origins=2 -lpthread -laio -o aio-stress aio-stress.c
  3. Run ./aio-stress -m -s 2MB

Expected results: Test to run without error?

Actual result:

$ ./aio-stress -m -s 2MB file size 1024MB, record size 64KB, depth 64, ios per iteration 8 max io_submit 8, buffer alignment set to 4KB threads 1 files 1 contexts 1 context offset 2MB verification off Running single thread version ==20656==WARNING: MemorySanitizer: use-of-uninitialized-value [...]

How reproducible is the problem? The problem can be reproduced every time.

Version information: Clang 3.8 Ubuntu 14.04 x86_64

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/sanitizers/issues/688, or mute the thread https://github.com/notifications/unsubscribe/AAZuSkol1F9dFxijdSmnJka5n7VioNgrks5qKmvCgaJpZM4Izf8U .

sitsofe commented 8 years ago

Thanks for the advice - the attached patch based on your suggestion seems to solve the issue on an x86_64 Linux after building and linking appropriately: libaio-san.txt

I guess this issue should be closed but before then is there anything that can be done more generically if users are willing to deal with false negatives (e.g. marking all memory from functions in a given given library as unpoisoned)?

yugr commented 8 years ago

Also, perhaps MSan could warn on unrecognized asm blocks?

sitsofe commented 8 years ago

@yugr: That could create a lot of spurious warnings because you would never know if a given asm block was one you should care about. In my case I needed to care because asm was being used to run syscalls but perhaps the asm is putting values in allocated memory / making the memory invalid etc behind the scenes. It would be very hard to make something generic that wouldn't just trigger on all asm blocks...

sitsofe commented 8 years ago

(http://thread.gmane.org/gmane.comp.compilers.llvm.cvs/167562/focus=167565 talks about why libaio calls aren't intercepted because doing so would force every *san program to use -laio)

yugr commented 8 years ago

It would be very hard to make something generic that wouldn't just trigger on all asm blocks...

But isn't this desired? I mean 99% of asm blocks need special handling in MSan.

eugenis commented 8 years ago

In my experience it is a lot less than 99%. A lot of assembly blocks don't have any memory side effects (like rdtsc or cpuid), and a lot have their memory effects declared in the constraints (MSan can handle those). Warning indiscriminately on every asm block will break projects that use -Werror.

As for a more generic solution, that's hard. I don't know how to define "all memory from functions in a given library" - especially when this library is not instrumented.

On Mon, Jun 13, 2016 at 4:09 AM, Yury Gribov notifications@github.com wrote:

It would be very hard to make something generic that wouldn't just trigger on all asm blocks...

But isn't this desired? I mean 99% of asm blocks need special handling in MSan.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/sanitizers/issues/688#issuecomment-225552126, or mute the thread https://github.com/notifications/unsubscribe/AAZuSk_Ikzs-oVrRlt4vfy-21GjyLDuPks5qLTpqgaJpZM4Izf8U .

sitsofe commented 8 years ago

@eugenis: I don't know how to define "all memory from functions in a given library" - especially when this library is not instrumented.

Thank you for answering my remaining question. Marking this issue as closed.