Missing support for long double

[ramosian-glider]

Originally reported on Google Code with ID 151

What steps will reproduce the problem?
1. Write a program using "long double" type with e.g. a heap-buffer-overflow error
(see below for an example).
2. Compile it with ASan and run it.
3. Observe no ASan report.

What is the expected output? What do you see instead?
Expected an ASan report on a heap-buffer-overflow (it is displayed for ints/floats/doubles/__mm128).
There was no ASan report (i.e. it didn't get detected).

What version of the product are you using? On what operating system?
Clang 3.2 on Ubuntu 12.10.

Please provide any additional information below.
Example test + output:
> cat test2.cpp
#include <xmmintrin.h>
int main(void) {  TYPE *x = new TYPE[10]; x[15] = VALUE; }

> clang test2.cpp -fsanitize=address -fno-omit-frame-pointer -g "-DTYPE=long double"
-DVALUE=1234.0
> ./a.out
>

As you see no ASan report was generated. If I set TYPE/VALUE to anything else (I tested
int, float, double, __mm128) it works OK and ASan correctly shows the heap-buffer-overflow
report. So it's just the "long double" type.

Reported by gynvael@google.com on 2013-02-06 10:32:31

[ramosian-glider]

Few notes: 
x86_64: sizeof 16, alignment 16
i386: sizeof 12, alignment 4

So, on i386 we will need to instrument long double accesses as 3 (or 2!) 4-byte accesses.

On x86_64 we can do it with one 16-bit access. 
gcc already does that: 
% cat long_double.cc 
long double a[10];
void foo(int i) {
  a[i] = 1;
}
% gcc  -fsanitize=address long_double.cc -O2 -S -o - -m64 | grep __asan_report
    call    __asan_report_store16
% gcc  -fsanitize=address long_double.cc -O2 -S -o - -m32 | grep __asan_report
% 

Reported by konstantin.s.serebryany on 2013-02-06 10:52:39

[ramosian-glider]

http://llvm.org/viewvc/llvm-project?rev=175266&view=rev 
implements long double support in 64-bit

I am not sure if 32-bit long double is worth the trouble

Reported by konstantin.s.serebryany on 2013-02-15 12:49:35

[ramosian-glider]

That solution was wrong, reverted in 
http://llvm.org/viewvc/llvm-project?rev=175442&view=rev .

Reported by konstantin.s.serebryany on 2013-02-18 13:48:01

[ramosian-glider]

Second attempt: 
 http://llvm.org/viewvc/llvm-project?rev=175507&view=rev
 http://llvm.org/viewvc/llvm-project?rev=175508&view=rev

Now both 32- and 64-bit is fixed. 

Reported by konstantin.s.serebryany on 2013-02-19 11:48:11

• Status changed: Fixed

[ramosian-glider]

Adding Project:AddressSanitizer as part of GitHub migration.

Reported by ramosian.glider on 2015-07-30 09:13:40

• Labels added: ProjectAddressSanitizer