Do not sanitize kernel area on 32-bit targets

[GoogleCodeExporter]

As suggested in 
https://groups.google.com/forum/#!topic/address-sanitizer/aX4J7H36QDk it may 
make sense to avoid mmaping shadow memory for kernel area ( (0xc0000000 - 
0xffffffff) on 32-bit Linux targets. This would save us 128M of virtual 
adresses.

Original issue reported on code.google.com by tetra2...@gmail.com on 19 Mar 2014 at 3:49

[GoogleCodeExporter]

Posted draft commit in http://llvm-reviews.chandlerc.com/D3119

Original comment by tetra2...@gmail.com on 19 Mar 2014 at 3:52

[GoogleCodeExporter]

64-bit app on x86_64 machine:
|| `[0x10007fff8000, 0x7fffffffffff]` || HighMem    ||
|| `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
|| `[0x00008fff7000, 0x02008fff6fff]` || ShadowGap  ||
|| `[0x00007fff8000, 0x00008fff6fff]` || LowShadow  ||
|| `[0x000000000000, 0x00007fff7fff]` || LowMem     ||

32-bit app on x86_64 machine:
|| `[0x40000000, 0xffffffff]` || HighMem    ||
|| `[0x28000000, 0x3fffffff]` || HighShadow ||
|| `[0x24000000, 0x27ffffff]` || ShadowGap  ||
|| `[0x20000000, 0x23ffffff]` || LowShadow  ||
|| `[0x00000000, 0x1fffffff]` || LowMem     ||

32-bit app on i686 machine:
|| `[0x38000000, 0xbfffffff]` || HighMem    ||
|| `[0x27000000, 0x37ffffff]` || HighShadow ||
|| `[0x24000000, 0x26ffffff]` || ShadowGap  ||
|| `[0x20000000, 0x23ffffff]` || LowShadow  ||
|| `[0x00000000, 0x1fffffff]` || LowMem     ||

check-asan and check-sanitizer tests passed (apart from non-portable tests).

Original comment by tetra2...@gmail.com on 19 Mar 2014 at 3:54

[GoogleCodeExporter]

should be fixed by http://llvm.org/viewvc/llvm-project?rev=204897&view=rev

Original comment by konstant...@gmail.com on 27 Mar 2014 at 3:03

• Changed state: Fixed

[GoogleCodeExporter]

fix reverted in http://llvm.org/viewvc/llvm-project?rev=207768&view=rev

Original comment by konstant...@gmail.com on 1 May 2014 at 4:35

• Changed state: New

[GoogleCodeExporter]

Ok, how about this - try mapping 1 page in the middle of high Gb and see if 
resulting addresss managed to get there.

Original comment by tetra2...@gmail.com on 5 May 2014 at 6:00

[GoogleCodeExporter]

Another option would be to scan /proc/PID/maps for read-write regions in high 
addresses but this sounds somewhat less robust...

Original comment by tetra2...@gmail.com on 5 May 2014 at 12:36

[GoogleCodeExporter]

We can try 
  1. checking /proc/PID/maps
  2. mmapping several addresses in the upper 1Gb
  3. checking where is the current stack pointer

None of that is exceptionally robust (although #3 should work on most systems 
I've encountered). 
Maybe also hide this feature under a flag?

Original comment by konstant...@gmail.com on 6 May 2014 at 11:49

[GoogleCodeExporter]

I'd say mmaping the last page in address space and checking the output sounds 
pretty robust. I'm ok with flag though.

Original comment by tetra2...@gmail.com on 6 May 2014 at 12:34

[GoogleCodeExporter]

>> mmaping the last page
This is a subset of #2

Original comment by konstant...@gmail.com on 6 May 2014 at 12:50

[GoogleCodeExporter]

Why do you suggest several mappings btw? AFAIK mmap will try to map the closest 
page to the requested one.

Original comment by tetra2...@gmail.com on 6 May 2014 at 1:00

[GoogleCodeExporter]

ah, you suggest to use a hint, not MAP_FIXED. 
May work. Especially if you hide it under a flag :) 

Original comment by konstant...@gmail.com on 6 May 2014 at 1:49

[GoogleCodeExporter]

Ok, I'll send a patch then. As for flags - as I said I personally have no 
problem with them but this feature looks really important. Perhaps we could 
enable it by default when it's proven stable?

Original comment by tetra2...@gmail.com on 6 May 2014 at 2:32

[GoogleCodeExporter]

We can have it under a flag which is *on* by default,
but it's important to be able to disable it if it goes wrong somewhere

Original comment by konstant...@gmail.com on 6 May 2014 at 2:36

[GoogleCodeExporter]

Ok, mmap trick didn't work for (it looks like kernel does not respect address 
hint deeply enough). All other tricks are rather unstable as well.

I think the core problem with schroot is that it sets non-trivial personality. 
We should be able to detect this (by checking return value of personality(-1)) 
and leave default 4G limit unchanged.

Original comment by tetra2...@gmail.com on 8 May 2014 at 12:15

[GoogleCodeExporter]

Done in r204897 and r208760.

Original comment by tetra2...@gmail.com on 17 Jun 2014 at 5:12

• Changed state: Done

[GoogleCodeExporter]

Adding Project:AddressSanitizer as part of GitHub migration.

Original comment by ramosian.glider@gmail.com on 30 Jul 2015 at 9:14

• Added labels: ProjectAddressSanitizer