Missing support for long double

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Write a program using "long double" type with e.g. a heap-buffer-overflow 
error (see below for an example).
2. Compile it with ASan and run it.
3. Observe no ASan report.

What is the expected output? What do you see instead?
Expected an ASan report on a heap-buffer-overflow (it is displayed for 
ints/floats/doubles/__mm128). There was no ASan report (i.e. it didn't get 
detected).

What version of the product are you using? On what operating system?
Clang 3.2 on Ubuntu 12.10.

Please provide any additional information below.
Example test + output:
> cat test2.cpp
#include <xmmintrin.h>
int main(void) {  TYPE *x = new TYPE[10]; x[15] = VALUE; }

> clang test2.cpp -fsanitize=address -fno-omit-frame-pointer -g "-DTYPE=long 
double" -DVALUE=1234.0
> ./a.out
>

As you see no ASan report was generated. If I set TYPE/VALUE to anything else 
(I tested int, float, double, __mm128) it works OK and ASan correctly shows the 
heap-buffer-overflow report. So it's just the "long double" type.

Original issue reported on code.google.com by gynv...@google.com on 6 Feb 2013 at 10:32

[GoogleCodeExporter]

Few notes: 
x86_64: sizeof 16, alignment 16
i386: sizeof 12, alignment 4

So, on i386 we will need to instrument long double accesses as 3 (or 2!) 4-byte 
accesses. 
On x86_64 we can do it with one 16-bit access. 
gcc already does that: 
% cat long_double.cc 
long double a[10];
void foo(int i) {
  a[i] = 1;
}
% gcc  -fsanitize=address long_double.cc -O2 -S -o - -m64 | grep __asan_report
    call    __asan_report_store16
% gcc  -fsanitize=address long_double.cc -O2 -S -o - -m32 | grep __asan_report
% 

Original comment by konstant...@gmail.com on 6 Feb 2013 at 10:52

[GoogleCodeExporter]

http://llvm.org/viewvc/llvm-project?rev=175266&view=rev 
implements long double support in 64-bit

I am not sure if 32-bit long double is worth the trouble

Original comment by konstant...@gmail.com on 15 Feb 2013 at 12:49

[GoogleCodeExporter]

That solution was wrong, reverted in 
http://llvm.org/viewvc/llvm-project?rev=175442&view=rev .

Original comment by konstant...@gmail.com on 18 Feb 2013 at 1:48

[GoogleCodeExporter]

Second attempt: 
 http://llvm.org/viewvc/llvm-project?rev=175507&view=rev
 http://llvm.org/viewvc/llvm-project?rev=175508&view=rev

Now both 32- and 64-bit is fixed. 

Original comment by konstant...@gmail.com on 19 Feb 2013 at 11:48

• Changed state: Fixed