[GCC] ASan: mallo.c (mallopt): fails to set malloc params with ASan enabled

[ramosian-glider]

Originally reported on Google Code with ID 143

What steps will reproduce the problem?

cat << \EOF > test.c
#include <malloc.h>
#include <stdio.h>

int main(void) {
  int ret = 0;
  if ((ret = mallopt(M_MMAP_MAX, 262144)) < 0) {
    printf("Failed to set M_MMAP_MAX Return: %d\n", ret);
  }
  if ((ret = mallopt(M_TRIM_THRESHOLD, 524288)) < 0) {
    printf("Failed to set M_TRIM_THRESHOLD Return: %d\n", ret);
  }
  if ((ret = mallopt(M_TOP_PAD, 5242880)) < 0) {
    printf("Failed to set M_TOP_PAD Return: %d\n", ret);
  }
  if ((ret = mallopt(M_MMAP_THRESHOLD, 131072)) < 0) {
    printf("Failed to set M_MMAP_THRESHOLD Return: %d\n", ret);
  }
  return 0;
}
EOF

$ gcc test.c
$ ./a.out
# prints nothing. ``mallopt'' returns here 1 for all params. 
$ gcc -fsanitize=address -fno-omit-frame-pointer -Os -g test.c
$ ./a.out
Failed to set M_MMAP_MAX Return: -1
Failed to set M_TRIM_THRESHOLD Return: -1
Failed to set M_TOP_PAD Return: -1
Failed to set M_MMAP_THRESHOLD Return: -1
# ``mallopt'' returns -1 for each param
# only removing -fsanitize=address makes it work again

What is the expected output? What do you see instead?
Is the following expected? If so, could it be noted in GCC and probably LLVM/Clang
documentation?

What version of the product are you using? On what operating system?
GNU GCC 4.8.0, experimental, r195260, compiled with release checking. Scientific Linux
5.3

$ cat /etc/redhat-release
Scientific Linux CERN SLC release 5.8 (Boron)
# It's based on Red Hat Enterprise Linux

Please provide any additional information below.

Reported by David.Abdurachmanov on 2013-01-18 10:34:35

[ramosian-glider]

David, this is totally expected. 
asan uses a completely different memory allocator than the one in glibc 
and none if these mallopt options make sense for it. 

As for mentioning it in docs... Dunno. Sometimes I feel we have too many docs already.
Still: https://code.google.com/p/address-sanitizer/wiki/Incompatiblity

Reported by konstantin.s.serebryany on 2013-01-18 10:55:15

• Status changed: WontFix

[ramosian-glider]

Adding Project:AddressSanitizer as part of GitHub migration.

Reported by ramosian.glider on 2015-07-30 09:13:40

• Labels added: ProjectAddressSanitizer