search

389ds / 389-ds-base

The enterprise-class Open Source LDAP server for Linux
https://www.port389.org/
Other
213 stars 94 forks source link

nsslapd-mdb-max-dbs autotuning doesn't work properly #6374

Open progier389 opened 1 month ago

progier389 commented 1 month ago

Issue Description

[1] After restarting the server nsslapd-mdb-max-dbs may not be high enough to add a new backend because the value computation is wrong.

[2] dbscan fails to open the database if nsslapd-mdb-max-dbs has been increased.

[3] dbscan crash when closing the database (typically when using -S)

Package Version and Platform:

Steps to Reproduce Steps to reproduce the behavior:

  1. See testcase dirsrvtests/tests/suites/config/config_test.py::test_lmdb_autotuned_maxdbs in the PR

Expected results testcase should PASS libasan should only report leaks

Additional context

A run of the testcase with libasan howed several heap related issues:

   #0 0x7f5abe4c7d57 in import_abort_all /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/import.c:162
    #1 0x7f5abe4c7e1a in factory_destructor /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/import.c:195
    #2 0x7f5ac569ac4c in factory_destroy_extension /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/factory.c:366
    #3 0x42982a in connection_cleanup /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/connection.c:179
    #4 0x43565b in connection_table_move_connection_out_of_active_list /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/conntable.c:470
    #5 0x437fc7 in setup_pr_read_pds /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/daemon.c:1566
    #6 0x437fc7 in ct_list_thread /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/daemon.c:1352
    #7 0x7f5ac4e193b6 in _pt_root (/lib64/libnspr4.so+0x2b3b6) (BuildId: e6dfe24882ade9ec65f7f1591a3e9e249e5a5780)
    #8 0x7f5ac5e5df95 in asan_thread_start(void*) (/lib64/libasan.so.8+0x5df95) (BuildId: a4ad7eb954b390cf00f07fa10952988a41d9fc7a)
    #9 0x7f5ac52a66d6 in start_thread (/lib64/libc.so.6+0x976d6) (BuildId: 77c77fee058b19c6f001cf2cb0371ce3b8341211)
    #10 0x7f5ac532a60b in __clone3 (/lib64/libc.so.6+0x11b60b) (BuildId: 77c77fee058b19c6f001cf2cb0371ce3b8341211)

0x51d000242088 is located 2056 bytes inside of 2304-byte region [0x51d000241880,0x51d000242180)
freed by thread T29 here:
    #0 0x7f5ac5ef6638 in free.part.0 (/lib64/libasan.so.8+0xf6638) (BuildId: a4ad7eb954b390cf00f07fa10952988a41d9fc7a)
    #1 0x7f5ac565e56b in slapi_ch_free /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/ch_malloc.c:286
    #2 0x7f5abe5b848b in dbmdb_free_import_ctx /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_import_threads.c:4288
    #3 0x7f5abe5a17ef in dbmdb_public_dbmdb_import_main /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_import.c:1090
    #4 0x7f5abe5a23ee in dbmdb_import_main /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_import.c:1111
    #5 0x7f5ac4e193b6 in _pt_root (/lib64/libnspr4.so+0x2b3b6) (BuildId: e6dfe24882ade9ec65f7f1591a3e9e249e5a5780)
    #6 0x7f5ac5e5df95 in asan_thread_start(void*) (/lib64/libasan.so.8+0x5df95) (BuildId: a4ad7eb954b390cf00f07fa10952988a41d9fc7a)

previously allocated by thread T13 here:
    #0 0x7f5ac5ef7350 in calloc (/lib64/libasan.so.8+0xf7350) (BuildId: a4ad7eb954b390cf00f07fa10952988a41d9fc7a)
    #1 0x7f5ac565e183 in slapi_ch_calloc /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/ch_malloc.c:189
    #2 0x7f5abe5a891f in dbmdb_import_workerq_init /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_import_threads.c:361
    #3 0x7f5abe5b7e9f in dbmdb_import_init_writer /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_import_threads.c:4236
    #4 0x7f5abe5a4044 in dbmdb_bulk_import_start /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_import.c:1394
    #5 0x7f5abe5a4044 in dbmdb_ldbm_back_wire_import /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_import.c:1547
    #6 0x7f5abe4c7c00 in ldbm_back_wire_import /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/import.c:141
    #7 0x7f5ac565bf82 in process_bulk_import_op /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/bulk_import.c:132
    #8 0x7f5ac565c265 in slapi_start_bulk_import /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/bulk_import.c:48
    #9 0x7f5abe0ef731 in multisupplier_extop_StartNSDS50ReplicationRequest /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/plugins/replication/repl_extop.c:993
    #10 0x7f5ac5732d02 in plugin_call_exop_plugins /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/plugin.c:511
    #11 0x44372d in do_extended /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/extendop.c:445
    #12 0x43338f in connection_dispatch_operation /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/connection.c:708
    #13 0x43338f in connection_threadmain /home/progier/sb/i6361/tst/source/389-ds-base/ldap/servers/slapd/connection.c:1921
    #14 0x7f5ac4e193b6 in _pt_root (/lib64/libnspr4.so+0x2b3b6) (BuildId: e6dfe24882ade9ec65f7f1591a3e9e249e5a5780)
    #15 0x7f5ac5e5df95 in asan_thread_start(void*) (/lib64/libasan.so.8+0x5df95) (BuildId: a4ad7eb954b390cf00f07fa10952988a41d9fc7a)

    #0 0x7fa7d955ccad in add_dbi /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_instance.c:319
    #1 0x7fa7d9563b3b in add_index_dbi /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_instance.c:347
    #2 0x7fa7e07db684 in avl_inapply /home/progier/sb/i6374/tst/source/389-ds-base/ldap/libraries/libavl/avl.c:484
    #3 0x7fa7e07db6b4 in avl_inapply /home/progier/sb/i6374/tst/source/389-ds-base/ldap/libraries/libavl/avl.c:490
    #4 0x7fa7e07db6b4 in avl_inapply /home/progier/sb/i6374/tst/source/389-ds-base/ldap/libraries/libavl/avl.c:490
    #5 0x7fa7e07dd370 in avl_apply /home/progier/sb/i6374/tst/source/389-ds-base/ldap/libraries/libavl/avl.c:547
    #6 0x7fa7d9560db7 in dbmdb_open_all_files /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_instance.c:466
    #7 0x7fa7d9582f31 in dbmdb_instance_start /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/db-mdb/mdb_layer.c:243
    #8 0x7fa7d94a752a in dblayer_instance_start /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/dblayer.c:337
    #9 0x7fa7d94d5b6b in ldbm_instance_start /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/instance.c:261
    #10 0x7fa7d951673f in ldbm_instance_postadd_instance_entry_callback /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/back-ldbm/ldbm_instance_config.c:1055
    #11 0x7fa7e0675ed4 in dse_call_callback /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/dse.c:2722
    #12 0x7fa7e067dcbc in dse_add /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/dse.c:2463
    #13 0x7fa7e0643b3c in op_shared_add /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/add.c:759
    #14 0x7fa7e064635a in do_add /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/add.c:236
    #15 0x433087 in connection_dispatch_operation /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/connection.c:639
    #16 0x433087 in connection_threadmain /home/progier/sb/i6374/tst/source/389-ds-base/ldap/servers/slapd/connection.c:1921
    #17 0x7fa7dfd613b6 in _pt_root (/lib64/libnspr4.so+0x2b3b6) (BuildId: e6dfe24882ade9ec65f7f1591a3e9e249e5a5780)
    #18 0x7fa7e0e5df95 in asan_thread_start(void*) (/lib64/libasan.so.8+0x5df95) (BuildId: a4ad7eb954b390cf00f07fa10952988a41d9fc7a)
    #19 0x7fa7e02596d6 in start_thread (/lib64/libc.so.6+0x976d6) (BuildId: 77c77fee058b19c6f001cf2cb0371ce3b8341211)
    #20 0x7fa7e02dd60b in __clone3 (/lib64/libc.so.6+0x11b60b) (BuildId: 77c77fee058b19c6f001cf2cb0371ce3b8341211)