Handle assembly in sanitizer tools

[GoogleCodeExporter]

Sanitizer tools need to handle assembly (inline and not). For ASan and TSan 
that will improve tool coverage, for MSan - help avoid false positives.

ATM it seems like the best chance to do that is through some kind of MC pass in 
LLVM. AFAIK, those things (MC passes) do not exist yet. Making that happen 
would require significant infrastructure work.

Original issue reported on code.google.com by euge...@google.com on 6 Jun 2013 at 7:59

[GoogleCodeExporter]

Can we reason about some side effects of inline assembly pieces by looking at 
their constraints?

Original comment by ramosian.glider@gmail.com on 6 Jun 2013 at 9:00

[GoogleCodeExporter]

More or less. But this does not apply to .S assembly.

We already understand constraints line "r" or "q", but if there is a pointer to 
memory the best we can do in MSan is unpoison the entire allocator chunk it 
points to. The latter is not implemented yet, and it will not work for .S 
assembly anyway.

Original comment by euge...@google.com on 6 Jun 2013 at 9:02

[GoogleCodeExporter]

Issue 290 has been merged into this issue.

Original comment by konstant...@gmail.com on 7 May 2014 at 8:58

[GoogleCodeExporter]

BTW wouldn't it make more sense to decompile .S into LLVM and then sanitize 
result?

Original comment by tetra2...@gmail.com on 7 May 2014 at 5:31

[GoogleCodeExporter]

By handling inline assembly, we get .S for free - they all go through the same 
AsmParser.
.S -> .ll decompilation would be a very nice thing to have, but it sounds like 
a very complex task and I'm not aware of any reasonably good implementations. 
It would make even more sense to apply it to .o (to handle 
yasm/nasm/fortran/whatever-else-generated objects) and maybe even to .so (to 
handle system libraries, binary OpenGL blobs, etc).

.so adds yet another level of complexity.

Original comment by euge...@google.com on 8 May 2014 at 7:39

[GoogleCodeExporter]

> By handling inline assembly, we get .S for free -
> they all go through the same AsmParser.

I disagree here - dependencies and memory sizes in inline asm are explicit in 
asm constraints whereas in AsmParser you'll have to somehow devise them 
yourself.

> it sounds like a very complex task

Well, not much more complex as analyzing assembly and tracking data/control 
dependencies in AsmParser...

Original comment by tetra2...@gmail.com on 8 May 2014 at 9:01

[GoogleCodeExporter]

Asm constraints are not enough. We want to capture arbitrary memory accesses in 
asm (i.e. "memory" in the clobber list).

Original comment by euge...@google.com on 8 May 2014 at 9:11

[GoogleCodeExporter]

> We want to capture arbitrary memory accesses in asm
> (i.e. "memory" in the clobber list).

Interesting. Do you have some publicly available use-case?

Original comment by tetra2...@gmail.com on 8 May 2014 at 12:20

[GoogleCodeExporter]

Sure, /usr/include/bits/select.h:

# define __FD_ZERO(fdsp) \
  do {                                                                        \
    int __d0, __d1;                                                           \
    __asm__ __volatile__ ("cld; rep; " __FD_ZERO_STOS                         \
                          : "=c" (__d0), "=D" (__d1)                          \
                          : "a" (0), "0" (sizeof (fd_set)                     \
                                          / sizeof (__fd_mask)),              \
                            "1" (&__FDS_BITS (fdsp)[0])                       \
                          : "memory");                                        \
  } while (0)

This is causing false positives in MSan.

Original comment by euge...@google.com on 8 May 2014 at 12:25