search

ceph-dovecot / dovecot-ceph-plugin

Dovecot plugin for storing mails in a Ceph cluster
Other
131 stars 23 forks source link

Dictionary plugin unit test fail #237

Closed jrse closed 5 years ago

jrse commented 5 years ago

Timing issue unit tests.

Removed i_debug message from dictionary commit function.

=> This lead to unit test valgrind failure:

FAIL: it_test_dict_rados

==15196== Memcheck, a memory error detector ==15196== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==15196== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==15196== Command: /repo/src/tests/.libs/it_test_dict_rados ==15196== [==========] Running 3 tests from 1 test case. [----------] Global test environment set-up. [----------] 3 tests from DictTest [ RUN ] DictTest.init [ OK ] DictTest.init (1168 ms) [ RUN ] DictTest.iterate Debug: rados_dict_set(priv/A1, V-A1, oid=metadata) Debug: rados_dict_set(priv/A1/B1, V-A1/B1, oid=metadata) Debug: rados_dict_set(priv/A/B1/C1, V-A/B1/C1, oid=metadata) Debug: rados_dict_set(priv/A1/B1/C2, V-A1/B1/C2, oid=metadata) Debug: rados_dict_set(priv/A1/B2, V-A1/B2, oid=metadata) Debug: rados_dict_set(priv/A2, V-A2, oid=metadata) Debug: rados_dict_set(shared/S1, V-S1, oid=metadata) Debug: deploy_set_map: set_map size = 7 Debug: deploy_set_map_value: V-A/B1/C1 , oid=metadata Debug: deploy_set_map_value: V-A1 , oid=metadata Debug: deploy_set_map_value: V-A1/B1 , oid=metadata Debug: deploy_set_map_value: V-A1/B1/C2 , oid=metadata Debug: deploy_set_map_value: V-A1/B2 , oid=metadata Debug: deploy_set_map_value: V-A2 , oid=metadata Debug: deploy_set_map_value: V-S1 , oid=metadata Debug: rados_dict_iterate_init(priv/A1/) Debug: rados_dict_iterate_init(shared/S1) Debug: rados_dict_iterate_init(): private query Debug: rados_dict_iterate_init(): private err=0(Success) Debug: rados_dict_iterate_init(): shared query Debug: rados_dict_iterate_init(): shared err=0(Success) Debug: rados_dict_iterate_init(): priv get_return_value() err=0(Success) ==15196== Thread 17 msgr-worker-0: ==15196== Invalid read of size 8 ==15196== at 0x53D5320: std::cxx11::_List_base<ceph::buffer::ptr, std::allocator >::_M_clear() (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x540A8AA: ceph::buffer::list::claim(ceph::buffer::list&, unsigned int) (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x5435A23: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x543DDDA: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x5441041: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x6992B4D: DispatchQueue::fast_dispatch(Message) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AA422F: AsyncConnection::process() (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB4947: EventCenter::process_events(int, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB9035: ??? (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x575A57E: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25) ==15196== by 0x5C456DA: start_thread (pthread_create.c:463) ==15196== by 0x5F7E88E: clone (clone.S:95) ==15196== Address 0x19b9a790 is 0 bytes inside a block of size 32 free'd ==15196== at 0x4C3123B: operator delete(void) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==15196== by 0x4E6B01B: __gnu_cxx::new_allocator<std::_List_node >::deallocate(std::_List_node, unsigned long) (new_allocator.h:125) ==15196== by 0x4E69CA3: std::allocator_traits<std::allocator<std::_List_node > >::deallocate(std::allocator<std::_List_node >&, std::_List_node*, unsigned long) (alloc_traits.h:462) ==15196== by 0x4E6767D: std::cxx11::_List_base<ceph::buffer::ptr, std::allocator >::_M_put_node(std::_List_node) (stl_list.h:387) ==15196== by 0x4E65818: std::cxx11::_List_base<ceph::buffer::ptr, std::allocator >::_M_clear() (list.tcc:80) ==15196== by 0x4E642AF: std::cxx11::_List_base<ceph::buffer::ptr, std::allocator >::~_List_base() (stl_list.h:442) ==15196== by 0x4E6267B: std::__cxx11::list<ceph::buffer::ptr, std::allocator >::~list() (stl_list.h:733) ==15196== by 0x4E628C1: ceph::buffer::list::~list() (buffer.h:351) ==15196== by 0x4E619D2: rados_dict_iterate_init (dict-rados.cpp:748) ==15196== by 0x118BBC: DictTest_iterate_Test::TestBody() (it_test_dict_rados.cpp:96) ==15196== by 0x14623B: void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test, void (testing::Test::)(), char const) (gtest.cc:2401) ==15196== by 0x140390: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test, void (testing::Test::)(), char const) (gtest.cc:2437) ==15196== by 0x12412B: testing::Test::Run() (gtest.cc:2473) ==15196== by 0x124A8F: testing::TestInfo::Run() (gtest.cc:2655) ==15196== by 0x125122: testing::TestCase::Run() (gtest.cc:2773) ==15196== by 0x12C1CC: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4673) ==15196== by 0x1473FC: bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl, bool (testing::internal::UnitTestImpl::)(), char const) (gtest.cc:2401) ==15196== by 0x141196: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl, bool (testing::internal::UnitTestImpl::)(), char const) (gtest.cc:2437) ==15196== by 0x12ACAA: testing::UnitTest::Run() (gtest.cc:4281) ==15196== by 0x1197AB: RUN_ALL_TESTS() (gtest.h:2237) ==15196== by 0x119141: main (it_test_dict_rados.cpp:120) ==15196== Block was alloc'd at ==15196== at 0x4C3017F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==15196== by 0x699A883: decode_message(CephContext, int, ceph_msg_header&, ceph_msg_footer&, ceph::buffer::list&, ceph::buffer::list&, ceph::buffer::list&, Connection) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AA2A36: AsyncConnection::process() (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB4947: EventCenter::process_events(int, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB9035: ??? (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x575A57E: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25) ==15196== by 0x5C456DA: start_thread (pthread_create.c:463) ==15196== by 0x5F7E88E: clone (clone.S:95) ==15196== {

Memcheck:Addr8 fun:_ZNSt7__cxx1110_List_baseIN4ceph6buffer3ptrESaIS3_EE8_M_clearEv fun:_ZN4ceph6buffer4list5claimERS1_j obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 fun:_ZN13DispatchQueue13fast_dispatchEP7Message fun:_ZN15AsyncConnection7processEv fun:_ZN11EventCenter14process_eventsEiPNSt6chrono8durationImSt5ratioILl1ELl1000000000EEEE obj:/usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0 obj:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25 fun:start_thread fun:clone } ==15196== Invalid read of size 8 ==15196== at 0x54072CB: ceph::buffer::ptr::release() (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x53D532B: std::__cxx11::_List_base >::_M_clear() (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x540A8AA: ceph::buffer::list::claim(ceph::buffer::list&, unsigned int) (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x5435A23: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x543DDDA: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x5441041: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x6992B4D: DispatchQueue::fast_dispatch(Message*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AA422F: AsyncConnection::process() (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB4947: EventCenter::process_events(int, std::chrono::duration >*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB9035: ??? (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x575A57E: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25) ==15196== by 0x5C456DA: start_thread (pthread_create.c:463) ==15196== by 0x5F7E88E: clone (clone.S:95) ==15196== Address 0x19b9a7a0 is 16 bytes inside a block of size 32 free'd ==15196== at 0x4C3123B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==15196== by 0x4E6B01B: __gnu_cxx::new_allocator >::deallocate(std::_List_node*, unsigned long) (new_allocator.h:125) ==15196== by 0x4E69CA3: std::allocator_traits > >::deallocate(std::allocator >&, std::_List_node*, unsigned long) (alloc_traits.h:462) ==15196== by 0x4E6767D: std::__cxx11::_List_base >::_M_put_node(std::_List_node*) (stl_list.h:387) ==15196== by 0x4E65818: std::__cxx11::_List_base >::_M_clear() (list.tcc:80) ==15196== by 0x4E642AF: std::__cxx11::_List_base >::~_List_base() (stl_list.h:442) ==15196== by 0x4E6267B: std::__cxx11::list >::~list() (stl_list.h:733) ==15196== by 0x4E628C1: ceph::buffer::list::~list() (buffer.h:351) ==15196== by 0x4E619D2: rados_dict_iterate_init (dict-rados.cpp:748) ==15196== by 0x118BBC: DictTest_iterate_Test::TestBody() (it_test_dict_rados.cpp:96) ==15196== by 0x14623B: void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2401) ==15196== by 0x140390: void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2437) ==15196== by 0x12412B: testing::Test::Run() (gtest.cc:2473) ==15196== by 0x124A8F: testing::TestInfo::Run() (gtest.cc:2655) ==15196== by 0x125122: testing::TestCase::Run() (gtest.cc:2773) ==15196== by 0x12C1CC: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4673) ==15196== by 0x1473FC: bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2401) ==15196== by 0x141196: bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2437) ==15196== by 0x12ACAA: testing::UnitTest::Run() (gtest.cc:4281) ==15196== by 0x1197AB: RUN_ALL_TESTS() (gtest.h:2237) ==15196== by 0x119141: main (it_test_dict_rados.cpp:120) ==15196== Block was alloc'd at ==15196== at 0x4C3017F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==15196== by 0x699A883: decode_message(CephContext*, int, ceph_msg_header&, ceph_msg_footer&, ceph::buffer::list&, ceph::buffer::list&, ceph::buffer::list&, Connection*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AA2A36: AsyncConnection::process() (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB4947: EventCenter::process_events(int, std::chrono::duration >*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB9035: ??? (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x575A57E: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25) ==15196== by 0x5C456DA: start_thread (pthread_create.c:463) ==15196== by 0x5F7E88E: clone (clone.S:95) ==15196== { Memcheck:Addr8 fun:_ZN4ceph6buffer3ptr7releaseEv fun:_ZNSt7__cxx1110_List_baseIN4ceph6buffer3ptrESaIS3_EE8_M_clearEv fun:_ZN4ceph6buffer4list5claimERS1_j obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 fun:_ZN13DispatchQueue13fast_dispatchEP7Message fun:_ZN15AsyncConnection7processEv fun:_ZN11EventCenter14process_eventsEiPNSt6chrono8durationImSt5ratioILl1ELl1000000000EEEE obj:/usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0 obj:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25 fun:start_thread fun:clone } ==15196== Invalid free() / delete / delete[] / realloc() ==15196== at 0x4C3123B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==15196== by 0x53D5336: std::__cxx11::_List_base >::_M_clear() (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x540A8AA: ceph::buffer::list::claim(ceph::buffer::list&, unsigned int) (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x5435A23: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x543DDDA: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x5441041: ??? (in /usr/lib/x86_64-linux-gnu/librados.so.2.0.0) ==15196== by 0x6992B4D: DispatchQueue::fast_dispatch(Message*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AA422F: AsyncConnection::process() (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB4947: EventCenter::process_events(int, std::chrono::duration >*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB9035: ??? (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x575A57E: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25) ==15196== by 0x5C456DA: start_thread (pthread_create.c:463) ==15196== by 0x5F7E88E: clone (clone.S:95) ==15196== Address 0x19b9a790 is 0 bytes inside a block of size 32 free'd ==15196== at 0x4C3123B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==15196== by 0x4E6B01B: __gnu_cxx::new_allocator >::deallocate(std::_List_node*, unsigned long) (new_allocator.h:125) ==15196== by 0x4E69CA3: std::allocator_traits > >::deallocate(std::allocator >&, std::_List_node*, unsigned long) (alloc_traits.h:462) ==15196== by 0x4E6767D: std::__cxx11::_List_base >::_M_put_node(std::_List_node*) (stl_list.h:387) ==15196== by 0x4E65818: std::__cxx11::_List_base >::_M_clear() (list.tcc:80) ==15196== by 0x4E642AF: std::__cxx11::_List_base >::~_List_base() (stl_list.h:442) ==15196== by 0x4E6267B: std::__cxx11::list >::~list() (stl_list.h:733) ==15196== by 0x4E628C1: ceph::buffer::list::~list() (buffer.h:351) ==15196== by 0x4E619D2: rados_dict_iterate_init (dict-rados.cpp:748) ==15196== by 0x118BBC: DictTest_iterate_Test::TestBody() (it_test_dict_rados.cpp:96) ==15196== by 0x14623B: void testing::internal::HandleSehExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2401) ==15196== by 0x140390: void testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2437) ==15196== by 0x12412B: testing::Test::Run() (gtest.cc:2473) ==15196== by 0x124A8F: testing::TestInfo::Run() (gtest.cc:2655) ==15196== by 0x125122: testing::TestCase::Run() (gtest.cc:2773) ==15196== by 0x12C1CC: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4673) ==15196== by 0x1473FC: bool testing::internal::HandleSehExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2401) ==15196== by 0x141196: bool testing::internal::HandleExceptionsInMethodIfSupported(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (gtest.cc:2437) ==15196== by 0x12ACAA: testing::UnitTest::Run() (gtest.cc:4281) ==15196== by 0x1197AB: RUN_ALL_TESTS() (gtest.h:2237) ==15196== by 0x119141: main (it_test_dict_rados.cpp:120) ==15196== Block was alloc'd at ==15196== at 0x4C3017F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==15196== by 0x699A883: decode_message(CephContext*, int, ceph_msg_header&, ceph_msg_footer&, ceph::buffer::list&, ceph::buffer::list&, ceph::buffer::list&, Connection*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AA2A36: AsyncConnection::process() (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB4947: EventCenter::process_events(int, std::chrono::duration >*) (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x6AB9035: ??? (in /usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0) ==15196== by 0x575A57E: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25) ==15196== by 0x5C456DA: start_thread (pthread_create.c:463) ==15196== by 0x5F7E88E: clone (clone.S:95) ==15196== { Memcheck:Free fun:_ZdlPv fun:_ZNSt7__cxx1110_List_baseIN4ceph6buffer3ptrESaIS3_EE8_M_clearEv fun:_ZN4ceph6buffer4list5claimERS1_j obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 obj:/usr/lib/x86_64-linux-gnu/librados.so.2.0.0 fun:_ZN13DispatchQueue13fast_dispatchEP7Message fun:_ZN15AsyncConnection7processEv fun:_ZN11EventCenter14process_eventsEiPNSt6chrono8durationImSt5ratioILl1ELl1000000000EEEE obj:/usr/lib/x86_64-linux-gnu/ceph/libceph-common.so.0 obj:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25 fun:start_thread fun:clone } Debug: rados_dict_iterate_init(): shared wait_for_complete_and_cb() err=0(Success) Debug: rados_dict_iterate_init(): shared get_return_value() err=0(Success) Debug: rados_dict_iterate_init(): r_val=0(Success) Debug: rados_dict_iterate_init(): r_val=0(Success) Debug: rados_dict_iterate_context a6e1102b9e62485c5c3b0000ef687694 - priv/A1/B1=V-A1/B1 Debug: rados_dict_iterate_context a6e1102b9e62485c5c3b0000ef687694 - priv/A1/B1/C2=V-A1/B1/C2 Debug: rados_dict_iterate_context a6e1102b9e62485c5c3b0000ef687694 - priv/A1/B2=V-A1/B2 Debug: rados_dict_iterate_context a6e1102b9e62485c5c3b0000ef687694 - shared/S1=V-S1 Debug: rados_dict_iterate() found key='priv/A1/B1', value='V-A1/B1' Debug: rados_dict_iterate() found key='priv/A1/B2', value='V-A1/B2' Debug: rados_dict_iterate() found key='shared/S1', value='V-S1' [ OK ] DictTest.iterate (816 ms) [ RUN ] DictTest.deinit [ OK ] DictTest.deinit (207 ms) [----------] 3 tests from DictTest (2202 ms total) [----------] Global test environment tear-down [==========] 3 tests from 1 test case ran. (5528 ms total) [ PASSED ] 3 tests. ==15196== ==15196== HEAP SUMMARY: ==15196== in use at exit: 6,917 bytes in 70 blocks ==15196== total heap usage: 52,002 allocs, 51,933 frees, 6,325,942 bytes allocated ==15196== ==15196== LEAK SUMMARY: ==15196== definitely lost: 0 bytes in 0 blocks ==15196== indirectly lost: 176 bytes in 2 blocks ==15196== possibly lost: 0 bytes in 0 blocks ==15196== still reachable: 6,709 bytes in 67 blocks ==15196== suppressed: 32 bytes in 1 blocks ==15196== Reachable blocks (those to which a pointer was found) are not shown. ==15196== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==15196== ==15196== For counts of detected and suppressed errors, rerun with: -v ==15196== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 1 from 1) FAIL it_test_dict_rados (exit status: 1)