WuBingzheng / libleak

detect memory leak by LD_PRELOAD, without changing the target program
249 stars 52 forks source link

Crash when loading #25

Closed J-cztery closed 1 year ago

J-cztery commented 1 year ago

Program terminated with signal 11, Segmentation fault.

0 0x0000000010e44cc0 in ?? ()

1 0x00002ba62f18b477 in malloc (size=18) at libleak.c:677

2 0x00002ba6340ecaaa in strdup () from /lib64/libc.so.6

3 0x00002ba62f18a274 in lib_maps_build () at libleak.c:168

4 0x00002ba62f18a565 in init () at libleak.c:342

5 0x00002ba62ef73973 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2

6 0x00002ba62ef6515a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2

WuBingzheng commented 1 year ago

Could you provide more information, such as your OS, cpu, and your program's programming language ? Besides, could you build a simple program(such as print "hello, world") and try it with libleak?

J-cztery commented 1 year ago

OS is Centos 7, CPU is KNL 7210, programming language is a mix of C/C++. In this case it is a binary which contains code for offload into KNC (Kinghts Corner coprocessors), optimized code for KNL and code that runs on NVIDA GPU (cuda).

The whole thing is compiled using two versions of icc and nvcc (11.2)

Clearly the original malloc's address it gets is wrong.

As for simple test i do not see expected behaviour either: 1 int main() { 2 int x = malloc(sizeof(int) 10); 3 4 } LD_PRELOAD=$PATH_TO_LIBLEAK test_malloc <-- does not return anything and i would expect something.

J-cztery commented 1 year ago

I figured the leak anyway and i doubt libleak would have found it because it was not a leak. It was mallocing memory but not touching all of it immediately, touching it as the job progressed, with RSS slowly increasing until it ran out of physical memory. This is due to overcommit configuration of our nodes.

WuBingzheng commented 1 year ago

You could try the following code:

int main() {
  int *x = malloc(10);  // allocate memory
  sleep(70); // libleak catches  the memory living longer than 60 seconds
  malloc(1);  // call malloc to give libleak a chance to run, and will log something 
  pause(); // wait for you to check out output log
}
ostosh commented 1 year ago

I ran into the same issue with a larger project that includes many dependencies. The cause is this map is fixed and not large enough: https://github.com/WuBingzheng/libleak/blob/master/libleak.c#L148

I suspect something like this will resolve the issue: https://github.com/WuBingzheng/libleak/pull/15

WuBingzheng commented 1 year ago

@ostosh Thanks, but he did not reply my comments. So I just change the array size from 100 to 1000, and draft a new release.