Memory leak in valueset sorted array

389-ds-bot commented 4 years ago

Cloned from Pagure issue: https://pagure.io/389-ds-base/issue/49061

Created at 2016-12-06 16:51:47 by tbordaz (@tbordaz)
Closed at 2017-02-16 15:49:27 as duplicate
Assigned to tbordaz (@tbordaz)

The attached test case reproduce several memory leaks during updates of large groups. The test case is using memberof plugin.

The leak is possibly related to patches of https://fedorahosted.org/389/ticket/48894

389-ds-bot commented 4 years ago

Comment from tbordaz (@tbordaz) at 2016-12-06 17:40:52

valgrind backstack

==13458== 24 bytes in 3 blocks are possibly lost in loss record 343 of 1,414
==13458==    at 0x4C28BF6: malloc (vg_replace_malloc.c:299)
==13458==    by 0x7419879: strdup (in /usr/lib64/libc-2.22.so)
==13458==    by 0x4E8D39F: slapi_ch_strdup (ch_malloc.c:204)
==13458==    by 0x4E84043: slapi_attr_init_locking_optional (attr.c:341)
==13458==    by 0x4E85617: attrlist_find_or_create_locking_optional (attrlist.c:56)
==13458==    by 0x4E8555C: attrlist_find_or_create (attrlist.c:37)
==13458==    by 0x4E85758: attrlist_merge_valuearray (attrlist.c:102)
==13458==    by 0x4EA5724: slapi_entry_attr_merge_sv (entry.c:2689)
==13458==    by 0x4EA56E1: slapi_entry_attr_merge (entry.c:2681)
==13458==    by 0x4EA5922: slapi_entry_attr_replace (entry.c:2772)
==13458==    by 0x4EA61D4: slapi_entry_attr_set_charptr (entry.c:3019)
==13458==    by 0x10F22F05: id2entry_add_ext (id2entry.c:162)
==13458==    by 0x10F63C15: ldbm_back_modify (ldbm_modify.c:706)
==13458==    by 0x4EDC063: op_shared_modify (modify.c:1055)
==13458==    by 0x4EDAC64: modify_internal_pb (modify.c:599)
==13458==    by 0x4EDA7E0: slapi_modify_internal_pb (modify.c:454)
==13458==    by 0x1189642E: memberof_add_memberof_attr (memberof.c:2934)                                              
==13458==    by 0x11896370: memberof_fix_memberof_callback (memberof.c:2902)
==13458==    by 0x11894448: memberof_modop_one_replace_r (memberof.c:1690)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x1189483B: memberof_mod_attr_list (memberof.c:1860)
==13458==    by 0x11894ABA: memberof_add_attr_list (memberof.c:1947)
==13458==    by 0x118938C1: memberof_postop_add (memberof.c:1291)

==13458== 48 bytes in 3 blocks are possibly lost in loss record 640 of 1,414
==13458==    at 0x4C28BF6: malloc (vg_replace_malloc.c:299)
==13458==    by 0x4E8D061: slapi_ch_malloc (ch_malloc.c:96)
==13458==    by 0x4F3B50E: ber_bvcpy (value.c:44)
==13458==    by 0x4F3BB97: slapi_value_set_berval (value.c:332)
==13458==    by 0x4F3B856: value_init (value.c:178)
==13458==    by 0x4F3B7DD: value_new (value.c:155)
==13458==    by 0x4F3B5D8: slapi_value_new_berval (value.c:87)
==13458==    by 0x4F3C7C9: valuearray_init_bervalarray (valueset.c:223)
==13458==    by 0x4E85B1B: attrlist_replace (attrlist.c:280)
==13458==    by 0x4EA7B58: entry_replace_values (entry.c:3908)
==13458==    by 0x4EA7727: entry_apply_mod (entry.c:3700)
==13458==    by 0x4EAAA1B: entry_apply_mods_wsi (entrywsi.c:1036)
==13458==    by 0x10F62A87: modify_apply_check_expand (ldbm_modify.c:247)
==13458==    by 0x10F63956: ldbm_back_modify (ldbm_modify.c:639)
==13458==    by 0x4EDC063: op_shared_modify (modify.c:1055)
==13458==    by 0x4EDAC64: modify_internal_pb (modify.c:599)
==13458==    by 0x4EDA7E0: slapi_modify_internal_pb (modify.c:454)
==13458==    by 0x1189642E: memberof_add_memberof_attr (memberof.c:2934)                                              
==13458==    by 0x11896370: memberof_fix_memberof_callback (memberof.c:2902)
==13458==    by 0x11894448: memberof_modop_one_replace_r (memberof.c:1690)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x1189483B: memberof_mod_attr_list (memberof.c:1860)
==13458==    by 0x11894ABA: memberof_add_attr_list (memberof.c:1947)
==13458==    by 0x118938C1: memberof_postop_add (memberof.c:1291)

==13458== 48 bytes in 3 blocks are possibly lost in loss record 641 of 1,414
==13458==    at 0x4C28BF6: malloc (vg_replace_malloc.c:299)
==13458==    by 0x4E8D061: slapi_ch_malloc (ch_malloc.c:96)
==13458==    by 0x4F3E771: slapi_valueset_add_attr_valuearray_ext (valueset.c:1164)
==13458==    by 0x4F3DDE6: slapi_valueset_add_valuearray (valueset.c:892)
==13458==    by 0x4E85779: attrlist_merge_valuearray (attrlist.c:103)
==13458==    by 0x4EA5724: slapi_entry_attr_merge_sv (entry.c:2689)
==13458==    by 0x4EA56E1: slapi_entry_attr_merge (entry.c:2681)
==13458==    by 0x4EA5922: slapi_entry_attr_replace (entry.c:2772)
==13458==    by 0x4EA61D4: slapi_entry_attr_set_charptr (entry.c:3019)
==13458==    by 0x10F22F05: id2entry_add_ext (id2entry.c:162)
==13458==    by 0x10F63C15: ldbm_back_modify (ldbm_modify.c:706)
==13458==    by 0x4EDC063: op_shared_modify (modify.c:1055)
==13458==    by 0x4EDAC64: modify_internal_pb (modify.c:599)
==13458==    by 0x4EDA7E0: slapi_modify_internal_pb (modify.c:454)
==13458==    by 0x1189642E: memberof_add_memberof_attr (memberof.c:2934)
==13458==    by 0x11896370: memberof_fix_memberof_callback (memberof.c:2902)
==13458==    by 0x11894448: memberof_modop_one_replace_r (memberof.c:1690)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x1189483B: memberof_mod_attr_list (memberof.c:1860)
==13458==    by 0x11894ABA: memberof_add_attr_list (memberof.c:1947)
==13458==    by 0x118938C1: memberof_postop_add (memberof.c:1291)

==13458== 56 bytes in 1 blocks are possibly lost in loss record 708 of 1,414
==13458==    at 0x4C28BF6: malloc (vg_replace_malloc.c:299)
==13458==    by 0x4E8D061: slapi_ch_malloc (ch_malloc.c:96)
==13458==    by 0x4F1FADD: slapi_new_rwlock (slapi2nspr.c:191)
==13458==    by 0x4F05545: pw_entry_constructor (pw.c:2564)
==13458==    by 0x4EACF8D: factory_create_extension (factory.c:319)
==13458==    by 0x4EA4572: slapi_entry_alloc (entry.c:1958)
==13458==    by 0x4EA4A75: slapi_entry_dup (entry.c:2104)
==13458==    by 0x10F08721: backentry_dup (backentry.c:85)
==13458==    by 0x10F6372C: ldbm_back_modify (ldbm_modify.c:600)
==13458==    by 0x4EDC063: op_shared_modify (modify.c:1055)
==13458==    by 0x4EDAC64: modify_internal_pb (modify.c:599)
==13458==    by 0x4EDA7E0: slapi_modify_internal_pb (modify.c:454)
==13458==    by 0x1189642E: memberof_add_memberof_attr (memberof.c:2934)
==13458==    by 0x11896370: memberof_fix_memberof_callback (memberof.c:2902)
==13458==    by 0x11894448: memberof_modop_one_replace_r (memberof.c:1690)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x11894337: memberof_modop_one_replace_r (memberof.c:1649)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x1189483B: memberof_mod_attr_list (memberof.c:1860)
==13458==    by 0x11894ABA: memberof_add_attr_list (memberof.c:1947)
==13458==    by 0x118938C1: memberof_postop_add (memberof.c:1291)

...

==13458== 24,804 bytes in 608 blocks are possibly lost in loss record 1,382 of 1,414
==13458==    at 0x4C28BF6: malloc (vg_replace_malloc.c:299)
==13458==    by 0x7419879: strdup (in /usr/lib64/libc-2.22.so)
==13458==    by 0x4E8D39F: slapi_ch_strdup (ch_malloc.c:204)
==13458==    by 0x4E99241: slapi_sdn_get_ndn (dn.c:2380)
==13458==    by 0x4E995E4: slapi_sdn_compare (dn.c:2505)
==13458==    by 0x10F22ACA: id2entry_add_ext (id2entry.c:81)
==13458==    by 0x10F63C15: ldbm_back_modify (ldbm_modify.c:706)
==13458==    by 0x4EDC063: op_shared_modify (modify.c:1055)
==13458==    by 0x4EDAC64: modify_internal_pb (modify.c:599)
==13458==    by 0x4EDA7E0: slapi_modify_internal_pb (modify.c:454)
==13458==    by 0x1189642E: memberof_add_memberof_attr (memberof.c:2934)
==13458==    by 0x11896370: memberof_fix_memberof_callback (memberof.c:2902)
==13458==    by 0x11894448: memberof_modop_one_replace_r (memberof.c:1690)
==13458==    by 0x11893C19: memberof_modop_one_r (memberof.c:1434)
==13458==    by 0x11894A1B: memberof_mod_attr_list_r (memberof.c:1921)
==13458==    by 0x1189483B: memberof_mod_attr_list (memberof.c:1860)
==13458==    by 0x11894ABA: memberof_add_attr_list (memberof.c:1947)
==13458==    by 0x118938C1: memberof_postop_add (memberof.c:1291)

389-ds-bot commented 4 years ago

Comment from tbordaz (@tbordaz) at 2016-12-06 18:20:56

attachment ticket49061_test.py

389-ds-bot commented 4 years ago

Comment from firstyear (@Firstyear) at 2016-12-21 12:55:50

Hi Thierry,

All those issues are originating in memberof code.

==13458== by 0x1189642E: memberof_add_memberof_attr (memberof.c:2934)

The entry added at line 2934 seems to be the culprit.

That's probably what needs freeing.

389-ds-bot commented 4 years ago

Comment from tbordaz (@tbordaz) at 2017-02-11 23:05:28

Metadata Update from @tbordaz:

Issue assigned to tbordaz
Issue set to the milestone: 1.3.6.0

389-ds-bot commented 4 years ago

Comment from tbordaz (@tbordaz) at 2017-02-16 15:47:00

This bug is a duplicate of #2125

The leak is reproducible up to 4973699 Ticket 47982 - Add CI test suite ds_logs

But is fixed by 0e48cdf Ticket 49066 - Memory leaks in server

closing it

389-ds-bot commented 4 years ago

Comment from tbordaz (@tbordaz) at 2017-02-16 15:47:10

Metadata Update from @tbordaz:

Issue close_status updated to: None

389-ds-bot commented 4 years ago

Comment from tbordaz (@tbordaz) at 2017-02-16 15:49:32

Metadata Update from @tbordaz:

Issue close_status updated to: duplicate
Issue status updated to: Closed (was: Open)

389ds / 389-ds-base

Memory leak in valueset sorted array #2120