search

honggyukim / heaptrace

a tool that collects and reports heap allocated memory
GNU General Public License v2.0
13 stars 3 forks source link

libheaptrace: Report self consuming memory #14

Open Bojun-Seo opened 1 year ago

Bojun-Seo commented 1 year ago

Knowing memory overhead can be important in some cases So add atomic variable to count size and report it

This patch is draft, not working yet.

Bojun-Seo commented 1 year ago

Current patch causes segfault. But I have no idea between this patch and utils::get_comm_name.

(gdb) r samples/a.out
... skip ...
(gdb) bt
#0  0x00007ffff7e405ce in std::_Rb_tree_decrement(std::_Rb_tree_node_base*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007ffff7fb7982 in std::_Rb_tree_iterator<std::pair<void* const, object_info_t> >::operator-- (this=<synthetic pointer>) at /usr/include/c++/11/bits/stl_tree.h:302
#2  std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_get_insert_unique_pos (this=this@entry=0x7ffff7fbc500 <addrmap>, __k=@0x55555556d0c0: 0x55555556b090) at /usr/include/c++/11/bits/stl_tree.h:2080
#3  0x00007ffff7fb7a6a in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_get_insert_hint_unique_pos (this=this@entry=0x7ffff7fbc500 <addrmap>, __position=..., __k=@0x55555556d0c0: 0x55555556b090) at /usr/include/c++/11/bits/stl_tree.h:2177
#4  0x00007ffff7fb6c01 in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<void* const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<void* const, object_info_t> >, std::piecewise_construct_t const&, std::tuple<void* const&>&&, std::tuple<>&&) (this=this@entry=0x7ffff7fbc500 <addrmap>, __pos=...) at /usr/include/c++/11/bits/stl_tree.h:2435
#5  0x00007ffff7fb7138 in std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::operator[] (__k=@0x7fffffffd758: 0x55555556b090,
    this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:350
#6  get_object (addr=<optimized out>, addr@entry=0x55555556b090) at /home/worker/heaptrace_pr/stacktrace.cc:411
#7  0x00007ffff7fb2664 in operator delete[] (ptr=0x55555556b090) at /home/worker/heaptrace_pr/libheaptrace.cc:208
#8  0x00007ffff7e8ee82 in std::basic_filebuf<char, std::char_traits<char> >::_M_destroy_internal_buffer() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x00007ffff7e922e7 in std::basic_filebuf<char, std::char_traits<char> >::close() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007ffff7fb85d2 in std::basic_filebuf<char, std::char_traits<char> >::~basic_filebuf (this=0x7fffffffda70, __in_chrg=<optimized out>) at /usr/include/c++/11/fstream:254
#11 std::basic_ifstream<char, std::char_traits<char> >::~basic_ifstream (this=0x7fffffffda60, __in_chrg=<optimized out>, __vtt_parm=<optimized out>) at /usr/include/c++/11/fstream:605
#12 utils::get_comm_name[abi:cxx11]() () at /home/worker/heaptrace_pr/utils.cc:43
#13 0x00007ffff7fb1fe7 in heaptrace_init () at /home/worker/heaptrace_pr/libheaptrace.cc:78
#14 0x00007ffff7fc947e in ?? () from /lib64/ld-linux-x86-64.so.2
#15 0x00007ffff7fc9568 in ?? () from /lib64/ld-linux-x86-64.so.2
#16 0x00007ffff7fe32ea in ?? () from /lib64/ld-linux-x86-64.so.2
#17 0x0000000000000001 in ?? ()
#18 0x00007fffffffe276 in ?? ()
#19 0x0000000000000000 in ?? ()
Bojun-Seo commented 1 year ago

To resolve this (crash) issue, I use C APIs on get_comm_name https://github.com/Bojun-Seo/heaptrace/commit/61889a3b6480e627253f75629e7913dda80d3554 It works normally with this patch. sample.c has same problem with: https://github.com/honggyukim/heaptrace/issues/17 factorial.c runs and reports normally.

But after I apply libheaptrace: Report self consuming memory patch on top. sample.c has same problem but factorial.c crashes on finalize. Followings are the status of the commits

3b20fca - Bojun Seo, 7 days ago : libheaptrace: Report self consuming memory - Thu Feb 16 19:02:29 2023 +0900
61889a3 - Bojun Seo, 15 minutes ago : utils: Use C APIs on utils::get_comm_name - Thu Feb 23 13:27:57 2023 +0900
703b2ea - Honggyu Kim, 7 weeks ago : heaptrace: Support mulitple sort keys - Thu Jan 5 20:42:26 2023 +0900
f1a90ab - Honggyu Kim, 6 weeks ago : utils:: Add utils::string_split function - Sat Jan 14 09:20:37 2023 +0900
... skip ...

And run heaptrace on gdb with factorial.out

$ gdb heaptrace
(gdb) r samples/factorial.out
[heaptrace] initialized for /proc/5414/maps (factorial.out)
[heaptrace]   finalized for /proc/5414/maps (factorial.out)
=================================================================
[heaptrace] dump allocation sorted by 'size' for /proc/5414/maps (factorial.out)
=== backtrace #1 === [count/peak: 1/1] [size/peak: 32 bytes/32 bytes] [age: 31.47 us]                                                                                                                                0 [0x7ffff7fb241f] operator new(unsigned long) +0x1f
 1 [0x7ffff7fb88b3] void std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11... +0x70
 2 [0x7ffff7fb869e] ./libheaptrace.so (+0x7ffff7fb869e)
 3 [0x7ffff7fb55e7] ./libheaptrace.so (+0x7ffff7fb55e7)
 4 [0x7ffff7fb1ea5] ./libheaptrace.so (+0x7ffff7fb1ea5)
 5 [0x7ffff7fc924e] /lib64/ld-linux-x86-64.so.2 (+0x7ffff7fc924e)
 6 [0x7ffff7dc1495] /lib/x86_64-linux-gnu/libc.so.6 (+0x7ffff7dc1495)
 7 [0x7ffff7dc1610] on_exit +0

=== backtrace #2 === [count/peak: 1/1] [size/peak: 10 bytes/10 bytes] [age: 80.33 us]
 0 [0x7ffff7fb269f] malloc +0x1f
 1 [0x5555555551c4] fac +0xe
 2 [0x5555555551d8] fac +0x13
 3 [0x5555555551d8] fac +0x13
 4 [0x5555555551d8] fac +0x13
 5 [0x5555555551d8] fac +0x13
 6 [0x5555555551d8] fac +0x13
 7 [0x5555555551d8] fac +0x13

... skip ...

=== backtrace #10 === [count/peak: 2/2] [size/peak: 1 bytes/1 bytes] [age: 103.700 us]
 0 [0x7ffff7fb269f] malloc +0x1f
 1 [0x5555555551a5] fac +0x7
 2 [0x5555555551d8] fac +0x13
 3 [0x5555555551d8] fac +0x13
 4 [0x5555555551d8] fac +0x13
 5 [0x5555555551d8] fac +0x13
 6 [0x5555555551d8] fac +0x13
 7 [0x5555555551d8] fac +0x13

[heaptrace] heap traced num of backtrace : 10
[heaptrace] heap traced allocation size  : 80 bytes
[heaptrace] allocator info (virtual)     : 135.168 KB
[heaptrace] allocator info (resident)    : 89.824 KB
[heaptrace] statm info (VSS/RSS/shared)  : 6.246 MB / 2.56 MB / 1.810 MB
[heaptrace] self consumed (memory overhead)  : 5280
=================================================================

Program received signal SIGSEGV, Segmentation fault.

0x00007ffff7c185f4 in std::_Rb_tree_decrement(std::_Rb_tree_node_base*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff7c185f4 in std::_Rb_tree_decrement(std::_Rb_tree_node_base*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007ffff7fb79a2 in std::_Rb_tree_iterator<std::pair<void* const, object_info_t> >::operator-- (this=<synthetic pointer>) at /usr/include/c++/11/bits/stl_tree.h:302
#2  std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_get_insert_h
int_unique_pos (this=this@entry=0x7ffff7fbc500 <addrmap>, __position=..., __k=@0x55555556e5e0: 0x55555556cf30) at /usr/include/c++/11/bits/stl_tree.h:2185
#3  0x00007ffff7fb6bb1 in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t
> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<void* const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<void* const, object_info_t> >, std::piecewise_construct_t const&, s
td::tuple<void* const&>&&, std::tuple<>&&) (this=this@entry=0x7ffff7fbc500 <addrmap>, __pos=...) at /usr/include/c++/11/bits/stl_tree.h:2435
#4  0x00007ffff7fb70e8 in std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::operator[] (__k=@0x7fffffffda88: 0x55555556cf30,
    this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:350
#5  get_object (addr=<optimized out>, addr@entry=0x55555556cf30) at /home/worker/heaptrace_pr/stacktrace.cc:413
#6  0x00007ffff7fb2524 in operator delete (ptr=0x55555556cf30) at /home/worker/heaptrace_pr/libheaptrace.cc:191
#7  operator delete (ptr=0x55555556cf30) at /home/worker/heaptrace_pr/libheaptrace.cc:186
#8  0x00007ffff7fb6beb in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > >::deallocate (__t=1, __p=<optimized out>, this=0x7ffff7fbc500 <addrmap>)
    at /usr/include/c++/11/ext/new_allocator.h:145
#9  std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > > >::deallocate (__n=1, __p=<optimized out>, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:496
#10 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_put_node (
    __p=<optimized out>, this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:565
#11 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_drop_node (
    __p=<optimized out>, this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:632
#12 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_emplace_hint
_unique<std::piecewise_construct_t const&, std::tuple<void* const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<void* const, object_info_t> >, std::piecewise_construct_t const&, std::tuple<void* const&
>&&, std::tuple<>&&) (this=this@entry=0x7ffff7fbc500 <addrmap>, __pos=...) at /usr/include/c++/11/bits/stl_tree.h:2440
#13 0x00007ffff7fb70e8 in std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::operator[] (__k=@0x7fffffffdb68: 0x55555556d250,
    this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:350
#14 get_object (addr=<optimized out>, addr@entry=0x55555556d250) at /home/worker/heaptrace_pr/stacktrace.cc:413
#15 0x00007ffff7fb2524 in operator delete (ptr=0x55555556d250) at /home/worker/heaptrace_pr/libheaptrace.cc:191
#16 operator delete (ptr=0x55555556d250) at /home/worker/heaptrace_pr/libheaptrace.cc:186
#17 0x00007ffff7fb35db in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > >::deallocate (__t=1, __p=0x55555556d250, this=<optimized out>)
    at /usr/include/c++/11/ext/new_allocator.h:132
#18 std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > > >::deallocate (__n=1, __p=0x55555556d250, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:496
#19 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_put_node (
    __p=0x55555556d250, this=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:565
#20 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_drop_node (
    __p=0x55555556d250, this=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:632
#21 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_erase (
    __x=0x55555556d250, this=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:1891
#22 0x00007ffff7fb711f in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t
> > >::_M_erase (this=<optimized out>, __x=0x55555556cdf0) at /usr/include/c++/11/bits/stl_tree.h:787
#23 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::~_Rb_tree (
    this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:984
#24 std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::~map (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/stl_map.h:302
#25 0x00007ffff7dc1a56 in __cxa_finalize () from /lib/x86_64-linux-gnu/libc.so.6
#26 0x00007ffff7fb2377 in __do_global_dtors_aux () from ./libheaptrace.so
#27 0x00007fffffffdd90 in ?? ()
Bojun-Seo commented 1 year ago

Now it works properly.

Report heaptrace self consuming memory overhead, for user who want to know memory overhead. Consider allocation requests when hook_guard is set are self allocation.

Add self consumed at the end of the report.

$ ./heaptrace samples/sample_leak.out
[heaptrace] initialized for /proc/6895/maps (sample_leak.out)
[heaptrace]   finalized for /proc/6895/maps (sample_leak.out)
=================================================================
[heaptrace] dump allocation sorted by 'size' for /proc/6895/maps (sample_leak.out)
=== backtrace #1 === [count/peak: 1/1] [size/peak: 4 bytes/4 bytes] [age: 193.682 us]
 0 [0x7f2638a0859f] malloc +0x1f
 1 [0x55baef5991bf] main +0x5
 2 [0x7f26387fbd90] /lib/x86_64-linux-gnu/libc.so.6 (+0x7f26387fbd90)
 3 [0x7f26387fbe40] __libc_start_main +0x20
 4 [0x55baef5990e5] _start +0x9

[heaptrace] heap traced num of backtrace : 1
[heaptrace] heap traced allocation size  : 4 bytes
[heaptrace] allocator info (virtual)     : 135.168 KB
[heaptrace] allocator info (resident)    : 78.368 KB
[heaptrace] statm info (VSS/RSS/shared)  : 6.250 MB / 3.891 MB / 3.661 MB
[heaptrace] self consumed (mem overhead) : 22.187 KB
=================================================================