search

ceph-dovecot / dovecot-ceph-plugin

Dovecot plugin for storing mails in a Ceph cluster
Other
131 stars 23 forks source link

Error occured while making storage-rbox. #374

Open cryptodriver opened 1 year ago

cryptodriver commented 1 year ago

Describe the bug Make failed. Error occured while making storage-rbox. Below is the detail.

source code is 1.0.0. os is Cent OS 7.

make[2]: Entering directory/root/dovecot-ceph-plugin-1.0.0/src/storage-rbox' /bin/sh ../../libtool --tag=CXX --mode=compile g++ -std=gnu++11 -DHAVE_CONFIG_H -I. -I../.. -I/usr/include/dovecot -I../../src/librmb -I../../src/librmb/tools/rmb -fno-strict-aliasing -Wall -Wextra -Wundef -Wwrite-strings -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wno-unused-parameter -Wno-missing-field-initializers -Wformat=2 -Wcast-align -Wformat-nonliteral -Wformat-security -Wsign-compare -Wstrict-aliasing -Wshadow -Winline -Wpacked -Wmissing-format-attribute -Wmissing-noreturn -Winit-self -Wmissing-include-dirs -Wunused-but-set-variable -Warray-bounds -Wreturn-type -Wno-overloaded-virtual -Wswitch-enum -Wswitch-default -Wno-undef -Wno-redundant-decls -Wno-error=unused-parameter -Wno-error=missing-field-initializers -Wno-error=overloaded-virtual -Wno-error=undef -Wno-error=redundant-decls -fpermissive -std=c++11 -fPIC -DPIC -MT lib10_doveadm_rbox_plugin_la-doveadm-rbox-plugin.lo -MD -MP -MF .deps/lib10_doveadm_rbox_plugin_la-doveadm-rbox-plugin.Tpo -c -o lib10_doveadm_rbox_plugin_la-doveadm-rbox-plugin.lo test -f 'doveadm-rbox-plugin.cpp' || echo './'doveadm-rbox-plugin.cpp libtool: compile: g++ -std=gnu++11 -DHAVE_CONFIG_H -I. -I../.. -I/usr/include/dovecot -I../../src/librmb -I../../src/librmb/tools/rmb -fno-strict-aliasing -Wall -Wextra -Wundef -Wwrite-strings -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wno-unused-parameter -Wno-missing-field-initializers -Wformat=2 -Wcast-align -Wformat-nonliteral -Wformat-security -Wsign-compare -Wstrict-aliasing -Wshadow -Winline -Wpacked -Wmissing-format-attribute -Wmissing-noreturn -Winit-self -Wmissing-include-dirs -Wunused-but-set-variable -Warray-bounds -Wreturn-type -Wno-overloaded-virtual -Wswitch-enum -Wswitch-default -Wno-undef -Wno-redundant-decls -Wno-error=unused-parameter -Wno-error=missing-field-initializers -Wno-error=overloaded-virtual -Wno-error=undef -Wno-error=redundant-decls -fpermissive -std=c++11 -fPIC -DPIC -MT lib10_doveadm_rbox_plugin_la-doveadm-rbox-plugin.lo -MD -MP -MF .deps/lib10_doveadm_rbox_plugin_la-doveadm-rbox-plugin.Tpo -c doveadm-rbox-plugin.cpp -fPIC -DPIC -o .libs/lib10_doveadm_rbox_plugin_la-doveadm-rbox-plugin.o In file included from doveadm-rbox-plugin.cpp:23:0: /usr/include/dovecot/str.h: In function 'const unsigned char str_data(const string_t)': /usr/include/dovecot/str.h:22:14: warning: invalid conversion from 'const void' to 'const unsigned char' [-fpermissive] return str->data; ^ In file included from doveadm-rbox-plugin.cpp:28:0: /usr/include/dovecot/mail-index-private.h: At global scope: /usr/include/dovecot/mail-index-private.h:268:44: warning: 'int mail_index_map(mail_index, mail_index_sync_handler_type)' hides constructor for 'struct mail_index_map' [-Wshadow] enum mail_index_sync_handler_type type); ^ In file included from ../../src/librmb/rados-storage.h:21:0, from ../../src/librmb/tools/rmb/rmb-commands.h:23, from doveadm-rbox-plugin.cpp:50: ../../src/librmb/rados-mail.h: In member function 'void librmb::RadosMail::set_lost_object(bool)': ../../src/librmb/rados-mail.h:91:45: warning: declaration of 'is_lost_object' shadows a member of 'this' [-Wshadow] void set_lost_object(bool is_lost_object) { lost_object = is_lost_object; } ^ doveadm-rbox-plugin.cpp: In function 'int cmd_rmb_create_ceph_index_run(doveadm_mail_cmd_context, mail_user)': doveadm-rbox-plugin.cpp:832:85: warning: format '%d' expects argument of type 'int', but argument 3 has type 'std::set<std::basic_string >::size_type {aka long unsigned int}' [-Wformat=] i_info("found %s mails in namespace %d",info->vname, mail_objects.size()); ^ doveadm-rbox-plugin.cpp:851:65: warning: format '%d' expects argument of type 'int', but argument 2 has type 'std::set<std::basic_string >::size_type {aka long unsigned int}' [-Wformat=] i_info("found %d mails in namespace",mail_objects.size()); ^ doveadm-rbox-plugin.cpp: In function 'int iterate_list_objects(mail_namespace, const mailbox_info, std::set<std::basic_string >&)': doveadm-rbox-plugin.cpp:880:63: error: 'mail_index_lock_sync' was not declared in this scope mail_index_lock_sync(box->index, "LOCKED_FOR_INDEX_CREATION"); ^ doveadm-rbox-plugin.cpp:882:112: error: too many arguments to function 'mailbox_transaction_context mailbox_transaction_begin(mailbox, mailbox_transaction_flags)' mailbox_transaction = mailbox_transaction_begin(box, MAILBOX_TRANSACTION_FLAG_EXTERNAL, "ceph_index_creation"); ^ In file included from /usr/include/dovecot/mail-storage-private.h:7:0, from /usr/include/dovecot/index-storage.h:5, from dovecot-all.h:28, from rbox-storage.h:21, from doveadm-rbox-plugin.cpp:58: /usr/include/dovecot/mail-storage.h:647:1: note: declared here mailbox_transaction_begin(struct mailbox box, ^ doveadm-rbox-plugin.cpp:909:62: error: 'mail_index_unlock' was not declared in this scope mail_index_unlock(box->index, "UNLOCKED_FOR_INDEX_CREATION"); ^ make[2]: [lib10_doveadm_rbox_plugin_la-doveadm-rbox-plugin.lo] Error 1 make[2]: Leaving directory `/root/dovecot-ceph-plugin-1.0.0/src/storage-rbox' make[1]: [install-recursive] Error 1 make[1]: Leaving directory /root/dovecot-ceph-plugin-1.0.0/src' make: *** [install-recursive] Error 1

jrse commented 1 year ago

what version of dovecot are you using?

cryptodriver commented 1 year ago

2.2.36

jrse commented 1 year ago

Try with 2.3.15 (tested) 2.2.x is somewhat deprecated and i didn't try it for a while.

cryptodriver commented 1 year ago

Try with 2.3.15 (tested) 2.2.x is somewhat deprecated and i didn't try it for a while.

Not work.

I have these needed modules installed on my OS.

dovecot 2.3.20 libtool.x86_64 2.4.2-22.el7_3 libtool-ltdl.x86_64 2.4.2-22.el7_3 jansson-devel.x86_64 2.10-1.el7 librados2-devel.x86_64 1:10.2.5-4.el7 dovecot23-devel.x86_64 1:2.3.20-1.gf.el7 redhat-rpm-config.noarch 9.1.0-88.el7.centos

Error was diffrent with the above one.

libtool: install: /bin/install -c .libs/rmb /usr/local/bin/rmb make[5]: Nothing to be done for install-data-am'. make[5]: Leaving directory/root/dovecot-ceph-plugin-1.0.0/src/librmb/tools/rmb' make[4]: Leaving directory /root/dovecot-ceph-plugin-1.0.0/src/librmb/tools/rmb' make[4]: Entering directory/root/dovecot-ceph-plugin-1.0.0/src/librmb/tools' make[5]: Entering directory /root/dovecot-ceph-plugin-1.0.0/src/librmb/tools' make[5]: Nothing to be done forinstall-exec-am'. make[5]: Nothing to be done for install-data-am'. make[5]: Leaving directory/root/dovecot-ceph-plugin-1.0.0/src/librmb/tools' make[4]: Leaving directory /root/dovecot-ceph-plugin-1.0.0/src/librmb/tools' make[3]: Leaving directory/root/dovecot-ceph-plugin-1.0.0/src/librmb/tools' make[2]: Leaving directory /root/dovecot-ceph-plugin-1.0.0/src/librmb' Making install in dict-rados make[2]: Entering directory/root/dovecot-ceph-plugin-1.0.0/src/dict-rados' depbase=echo libdict-rados-plugin.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||';\ /bin/sh ../../libtool --tag=CC --mode=compile gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -I/usr/include/dovecot -I../../src/librmb -fno-strict-aliasing -Wall -Wextra -Wundef -Wnested-externs -Wwrite-strings -Wpointer-arith -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wredundant-decls -Wno-unused-parameter -Wno-missing-field-initializers -Wdeclaration-after-statement -Wformat=2 -Wold-style-definition -Wcast-align -Wformat-nonliteral -Wformat-security -Wsign-compare -Wstrict-aliasing -Wshadow -Winline -Wpacked -Wmissing-format-attribute -Wmissing-noreturn -Winit-self -Wmissing-include-dirs -Wunused-but-set-variable -Warray-bounds -Wimplicit-function-declaration -Wreturn-type -Wswitch-enum -Wswitch-default -Wno-declaration-after-statement -Wno-error=unused-parameter -Wno-error=missing-field-initializers -Wno-error=declaration-after-statement -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fno-strict-aliasing -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mfunction-return=keep -mindirect-branch=keep -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I../.. -I/usr/include/dovecot -MT libdict-rados-plugin.lo -MD -MP -MF $depbase.Tpo -c -o libdict-rados-plugin.lo libdict-rados-plugin.c &&\ mv -f $depbase.Tpo $depbase.Plo libtool: compile: gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -I/usr/include/dovecot -I../../src/librmb -fno-strict-aliasing -Wall -Wextra -Wundef -Wnested-externs -Wwrite-strings -Wpointer-arith -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wredundant-decls -Wno-unused-parameter -Wno-missing-field-initializers -Wdeclaration-after-statement -Wformat=2 -Wold-style-definition -Wcast-align -Wformat-nonliteral -Wformat-security -Wsign-compare -Wstrict-aliasing -Wshadow -Winline -Wpacked -Wmissing-format-attribute -Wmissing-noreturn -Winit-self -Wmissing-include-dirs -Wunused-but-set-variable -Warray-bounds -Wimplicit-function-declaration -Wreturn-type -Wswitch-enum -Wswitch-default -Wno-declaration-after-statement -Wno-error=unused-parameter -Wno-error=missing-field-initializers -Wno-error=declaration-after-statement -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fno-strict-aliasing -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mfunction-return=keep -mindirect-branch=keep -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I../.. -I/usr/include/dovecot -MT libdict-rados-plugin.lo -MD -MP -MF .deps/libdict-rados-plugin.Tpo -c libdict-rados-plugin.c -fPIC -DPIC -o .libs/libdict-rados-plugin.o In file included from libdict-rados-plugin.c:15:0: dict-rados.h:19:24: error: macro "DOVECOT_PREREQ" requires 3 arguments, but only 2 given

if DOVECOT_PREREQ(2, 3)

                    ^

dict-rados.h:19:5: warning: "DOVECOT_PREREQ" is not defined [-Wundef]

if DOVECOT_PREREQ(2, 3)

 ^

dict-rados.h:31:24: error: macro "DOVECOT_PREREQ" requires 3 arguments, but only 2 given

if DOVECOT_PREREQ(2, 3)

                    ^

dict-rados.h:31:5: warning: "DOVECOT_PREREQ" is not defined [-Wundef]

if DOVECOT_PREREQ(2, 3)

 ^

dict-rados.h:47:24: error: macro "DOVECOT_PREREQ" requires 3 arguments, but only 2 given

if DOVECOT_PREREQ(2, 3)

                    ^

dict-rados.h:47:5: warning: "DOVECOT_PREREQ" is not defined [-Wundef]

if DOVECOT_PREREQ(2, 3)

 ^

libdict-rados-plugin.c:23:35: warning: initialization from incompatible pointer type [enabled by default] .wait = rados_dict_wait, ^ libdict-rados-plugin.c:23:35: warning: (near initialization for 'dict_driver_rados.v.wait') [enabled by default] libdict-rados-plugin.c:24:35: warning: initialization from incompatible pointer type [enabled by default] .lookup = rados_dict_lookup, ^ libdict-rados-plugin.c:24:35: warning: (near initialization for 'dict_driver_rados.v.lookup') [enabled by default] libdict-rados-plugin.c:25:35: warning: initialization from incompatible pointer type [enabled by default] .iterate_init = rados_dict_iterate_init, ^ libdict-rados-plugin.c:25:35: warning: (near initialization for 'dict_driver_rados.v.iterate_init') [enabled by default] libdict-rados-plugin.c:26:35: warning: initialization from incompatible pointer type [enabled by default] .iterate = rados_dict_iterate, ^ libdict-rados-plugin.c:26:35: warning: (near initialization for 'dict_driver_rados.v.iterate') [enabled by default] libdict-rados-plugin.c:27:35: warning: initialization from incompatible pointer type [enabled by default] .iterate_deinit = rados_dict_iterate_deinit, ^ libdict-rados-plugin.c:27:35: warning: (near initialization for 'dict_driver_rados.v.iterate_deinit') [enabled by default] libdict-rados-plugin.c:29:35: warning: initialization from incompatible pointer type [enabled by default] .transaction_commit = rados_dict_transaction_commit, ^ libdict-rados-plugin.c:29:35: warning: (near initialization for 'dict_driver_rados.v.transaction_commit') [enabled by default] libdict-rados-plugin.c:42:34: warning: initialization from incompatible pointer type [enabled by default] }}; ^ libdict-rados-plugin.c:42:34: warning: (near initialization for 'dict_driver_rados.v.lookup_async') [enabled by default] make[2]: [libdict-rados-plugin.lo] Error 1 make[2]: Leaving directory `/root/dovecot-ceph-plugin-1.0.0/src/dict-rados' make[1]: [install-recursive] Error 1 make[1]: Leaving directory `/root/dovecot-ceph-plugin-1.0.0/src' make: *** [install-recursive] Error 1

cryptodriver commented 1 year ago

Maybe this is a problem revelent with gcc version. which gcc version does this pluglin support?

jrse commented 1 year ago

use dovecot 2.3.15 the doveadm api changed in 2.3.20 this is a known problem.

cryptodriver commented 1 year ago

Much appreciate for your reply.

I have tried 2.3.15. Error was diffrent with above two. Although there were a lot of warnings, It seems that storage-rbox was built successfully. But making test failed. This is the detail of error.

・OS Environment dovecot.x86_64 2:2.3.15-1
dovecot-devel.x86_64 2:2.3.15-1
dovecot-lua.x86_64 2:2.3.15-1 jansson-devel.x86_64 2.10-1.el7
librados2.x86_64 1:10.2.5-4.el7
librados2-devel.x86_64 1:10.2.5-4.el7 libtool.x86_64 2.4.2-22.el7_3 libtool-ltdl.x86_64 2.4.2-22.el7_3 redhat-rpm-config.noarch 9.1.0-88.el7.centos

・error detail make[3]: Leaving directory '/root/dovecot-ceph-plugin-1.0.0/src/storage-rbox' make[2]: Leaving directory '/root/dovecot-ceph-plugin-1.0.0/src/storage-rbox' Making install in tests make[2]: Entering directory '/root/dovecot-ceph-plugin-1.0.0/src/tests' depbase=echo rmb/test_rmb.o | sed 's|[^/]*$|.deps/&|;s|\.o$||';\ g++ -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../../src/tests/mocks -I../../src/librmb -I../../src/dict-rados -I../../src/storage-rbox -I./googletest/googletest/include -I./googletest/googlemock/include -I/usr/include/dovecot -DGTEST_HAS_PTHREAD=1 -pthread -fpermissive -std=c++11 -Wmissing-declarations -fno-strict-aliasing -Wall -Wextra -Wundef -Wwrite-strings -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wno-unused-parameter -Wno-missing-field-initializers -Wformat=2 -Wcast-align -Wformat-nonliteral -Wformat-security -Wsign-compare -Wstrict-aliasing -Wshadow -Winline -Wpacked -Wmissing-format-attribute -Wmissing-noreturn -Winit-self -Wmissing-include-dirs -Wunused-but-set-variable -Warray-bounds -Wreturn-type -Wno-overloaded-virtual -Wswitch-enum -Wswitch-default -Wno-undef -Wno-redundant-decls -Wno-error=unused-parameter -Wno-error=missing-field-initializers -Wno-error=overloaded-virtual -Wno-error=undef -Wno-error=redundant-decls -fpermissive -std=c++11 -fPIC -DPIC -MT rmb/test_rmb.o -MD -MP -MF $depbase.Tpo -c -o rmb/test_rmb.o rmb/test_rmb.cpp &&\ mv -f $depbase.Tpo $depbase.Po cc1plus: warning: ./googletest/googletest/include: No such file or directory [-Wmissing-include-dirs] cc1plus: warning: ./googletest/googlemock/include: No such file or directory [-Wmissing-include-dirs] rmb/test_rmb.cpp:12:10: fatal error: gtest/gtest.h: No such file or directory 12 | #include "gtest/gtest.h" | ^~~~~~~ compilation terminated. make[2]: [Makefile:2648: rmb/test_rmb.o] Error 1 make[2]: Leaving directory '/root/dovecot-ceph-plugin-1.0.0/src/tests' make[1]: [Makefile:470: install-recursive] Error 1 make[1]: Leaving directory '/root/dovecot-ceph-plugin-1.0.0/src' make: *** [Makefile:523: install-recursive] Error 1

jrse commented 1 year ago

okay :-)

warning: ./googletest/googletest/include: No such file or directory [-Wmissing-include-dirs] cc1plus: warning: ./googletest/googlemock/include: No such file or directory [-Wmissing-include-dirs]

this means you didn't update the submodules, we are using google test as test framework.

you need to update the submodules with:

git submodule update --init --recursive

cryptodriver commented 1 year ago

okay :-)

warning: ./googletest/googletest/include: No such file or directory [-Wmissing-include-dirs] cc1plus: warning: ./googletest/googlemock/include: No such file or directory [-Wmissing-include-dirs]

this means you didn't update the submodules, we are using google test as test framework.

you need to update the submodules with:

git submodule update --init --recursive

Instead of using the released version of 1.0.0, I cloned source code from the master branch directly. And updated the submodules with the given command.

Then, tired again. This time built successfully. Thanks for your support.

My question is, this plugin does not support the latest version of dovecot(2.3.20 for now). Is there any plan to solve this problem?

jrse commented 1 year ago

glad to hear that it is working now.

yes, we are planning to make it work, i already created a issue in march for it https://github.com/ceph-dovecot/dovecot-ceph-plugin/issues/292. But for me it's a matter of available time to do it. I will add more details of what exactly needs to be done, it shouldn't be too complicated.

cryptodriver commented 1 year ago

glad to hear that it is working now.

yes, we are planning to make it work, i already created a issue in march for it #292. But for me it's a matter of available time to do it. I will add more details of what exactly needs to be done, it shouldn't be too complicated.

Thanks for your work. Wish to see some update.

cryptodriver commented 1 year ago

Error occured while writing to Ceph Cluster, any idea about this one?

Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: Thread::try_create(): pthread_create failed with error 11common/Thread.cc: In function 'void Thread::create(const char, size_t)' thread 7f45113d5740 time 2023-06-06 16:27:31.298147 Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: common/Thread.cc: 160: FAILED assert(ret == 0) Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367) Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 1: (()+0x15f695) [0x7f45044c0695] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 2: (()+0x182efa) [0x7f45044e3efa] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 3: (()+0x17bcbb) [0x7f45044dccbb] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 4: (()+0x17762f) [0x7f45044d862f] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 5: (()+0x5a23a) [0x7f45043bb23a] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 6: (rados_create2()+0x8c) [0x7f45043bf64c] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 7: (librmb::RadosClusterImpl::init(std::string const&, std::string const&)+0x82) [0x7f450dd0906e] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 8: (librmb::RadosStorageImpl::open_connection(std::string const&, std::string const&, std::string const&, std::string const&)+0x89) [0x7f450dd0d9f7] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 9: (rbox_open_rados_connection()+0x207) [0x7f450df7bd12] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 10: (rbox_mail_storage_copy(mail_save_context, mail)+0xab) [0x7f450df6cfa3] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 11: (rbox_mail_copy(mail_save_context, mail*)+0xd4) [0x7f450df6bfb8] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 12: (()+0x3d3a) [0x7f4510fbad3a] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 13: (()+0x4f26c) [0x7f4510cab26c] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 14: (mail_deliver_save()+0x1a9) [0x7f4510fbc349] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 15: (mail_deliver()+0x3f8) [0x7f4510fbcc08] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 16: (lmtp_local_default_deliver()+0x224) [0x5638cb242764] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 17: (lmtp_local_data()+0x485) [0x5638cb242dd5] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 18: (client_default_cmd_data()+0x18b) [0x5638cb24173b] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 19: (cmd_data_continue()+0x204) [0x5638cb2414d4] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 20: (()+0x5f7f4) [0x7f45109017f4] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 21: (io_loop_call_io()+0x65) [0x7f45109af425] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 22: (io_loop_handler_run_internal()+0x12b) [0x7f45109b0dab] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 23: (io_loop_handler_run()+0x59) [0x7f45109af529] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 24: (io_loop_run()+0x38) [0x7f45109af768] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 25: (master_service_run()+0x13) [0x7f45109193c3] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 26: (main()+0x20b) [0x5638cb23ff0b] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 27: (__libc_start_main()+0xf5) [0x7f45104f6555] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: 28: (()+0x6041) [0x5638cb240041] Jun 6 16:27:31 dev-host27 dovecot: lmtp: Error: NOTE: a copy of the executable, or objdump -rdS <executable> is needed to interpret this.

jrse commented 1 year ago

never tried this with Jewel 10.2.5. we tested with client and cluster > luminous looks like it cannot open connection to the ceph cluster.

common issues:

If you want, you can setup an evaluation environment by using the dovecot-ceph container i build to ease development and evaluation of new features. (centos7 and dovecot 2.3.15)

Here are some instructions

cryptodriver commented 1 year ago

Dovecot user does not have read access to /etc/ceph/*

Check OK.

is there a correct keyring (e.g. default is client.admin)

Check OK.

does the pool exist: e.g. default is mail_storage

No exists. But tested after created this manually. Still not work.

It's kind of weird. My ceph version is: ceph version 15.2.17 octopus (stable) Not 10.2.5. I have not 10.2.5 installed at all.

jrse commented 1 year ago

which librados-devel version is installed?

[root@bec9c4d1667a /]# yum list installed | grep librados librados-devel.x86_64 2:14.2.22-0.el7 @Ceph
librados2.x86_64 2:14.2.22-0.el7 @Ceph
libradosstriper1.x86_64 2:14.2.22-0.el7 @Ceph

cryptodriver commented 1 year ago

librados2-devel.x86_64 1:10.2.5-4.el7

Thanks for your reminding.

I have been using librados2-devel.x86_64 1:10.2.5-4.el7. Maybe that is the problem? This is what I have tried.

1) reinstall a newer one (librados). [root@test ~]# yum list installed | grep librados librados-devel.x86_64 2:15.2.17-0.el7 @ceph-noarch librados2.x86_64 2:15.2.17-0.el7 @ceph-noarch libradospp-devel.x86_64 2:15.2.17-0.el7 @ceph-noarch libradosstriper-devel.x86_64 2:15.2.17-0.el7 @ceph-noarch libradosstriper1.x86_64 2:15.2.17-0.el7 @ceph-noarch

2) rebuild & installed the plugin. 3) restart dovecot(reload this plugin)

This time another error occured. I know that this means can not connect ceph cluster. But I do have a right keyring and /etc/ceph/ceph.conf. Is there any mistake in my configuration file?

/etc/ceph/ceph.conf

[global] mon_host = 192.168.100.3 keyring = /etc/ceph/ceph.client.admin.keyring

/etc/dovecot/conf.d/90-plugin.conf

plugin { rbox_cluster_name = ceph rados_user_name = client.admin rbox_pool_name = mail_storage rbox_cfg_object_name = rbox_cfg rbox_bugfix_cephfs_21652 = false rados_save_log = /var/mail/rbox/save_file.log rados_check_empty_mailboxes= false rbox_ceph_aio_wait_for_safe_and_cb=false rbox_ceph_write_chunks=false }

Pool of mail_storage not exists, but it will be created by the plugin, I think.

<29315>: Error: Exception: setting up ceph connection: std::bad_alloc <29315>: Error: Open rados connection. Error(-1,Operation not permitted) (pool_name(mail_storage), cluster_name(ceph), rados_user_name(client.admin), alt_storage(0), alt_dir((null)) ) <29315>: Error: ERROR, cannot open rados connection (rbox_mail_storage_copy)
cryptodriver commented 1 year ago

Some update.

Ceph config seems no wrong. RMB command can connect Ceph Cluster normally.

I'm just curious about that the error is bad_alloc. That means something with memory.

If you want, you can setup an evaluation environment by using the dovecot-ceph container i build to ease development and evaluation of new features. (centos7 and dovecot 2.3.15) By the way, I tried this too. Not work...make failed..

jrse commented 1 year ago

Ok I will look at the container can u post some more details what went wrong? Regarding your exception it's happening when connecting to the cluster, can be it's the librados version maybe try with the one I posted. Will try it with your librados version.

jrse commented 1 year ago

there was a problem with the current branch, i already fixed this in the current dev version. for now please use bugfix/374-build-with-centos-container branch to work with dev container. Additionally i added some more logs, can u try again with the container and your env?

jrse commented 1 year ago

I think i found something, if you look at the centos librados-dev package: https://cbs.centos.org/koji/rpminfo?rpmID=389854 it does not have the *.hpp files included. think this i why i added them directly (from github src) in the centos container. So if the container works for you maybe you have the same issue, or old includes are still present.

cryptodriver commented 1 year ago

Tried with bugfix/374-build-with-centos-container branch. The same error occurred. Below is the detail of debug log.

Debug: [START] rbox-storage.cpp: rbox_storage_get_list_settings at line 88 Debug: [END] rbox-storage.cpp: rbox_storage_get_list_settings at line 101

Debug: [START] rbox-storage.cpp: rbox_storage_alloc at line 64 Debug: [END] rbox-storage.cpp: rbox_storage_alloc at line 83 Debug: [START] rbox-storage.cpp: rbox_storage_create at line 164 Debug: [END] rbox-storage.cpp: rbox_storage_create at line 175 Debug: [START] rbox-storage.cpp: rbox_mailbox_alloc at line 225 Debug: [START] rbox-storage.cpp: read_plugin_configuration at line 402 Debug: reading plugin conf: rados_check_empty_mailboxes=false Debug: reading plugin conf: rados_save_log=/var/mail/rbox/save_file.log Debug: reading plugin conf: rados_user_name=client.admin Debug: reading plugin conf: rbox_bugfix_cephfs_21652=false Debug: reading plugin conf: rbox_ceph_aio_wait_for_safe_and_cb=false Debug: reading plugin conf: rbox_ceph_write_chunks=false Debug: reading plugin conf: rbox_cfg_object_name=rbox_cfg Debug: reading plugin conf: rbox_chunk_size=(null) Debug: reading plugin conf: rbox_cluster_name=ceph Debug: reading plugin conf: rbox_index_pool_name=(null) Debug: reading plugin conf: rbox_object_search_method=(null) Debug: reading plugin conf: rbox_object_search_threads=(null) Debug: reading plugin conf: rbox_pool_name=mail_storage Debug: reading plugin conf: rbox_write_method=(null) Debug: [END] rbox-storage.cpp: read_plugin_configuration at line 423 Debug: [END] rbox-storage.cpp: rbox_mailbox_alloc at line 255

Debug: [START] rbox-storage.cpp: rbox_mailbox_open at line 642 Debug: [START] rbox-storage.cpp: rbox_mailbox_alloc_index at line 260 Debug: [END] rbox-storage.cpp: rbox_mailbox_alloc_index at line 276 Debug: [START] rbox-storage.cpp: rbox_open_mailbox at line 318 Debug: [END] rbox-storage.cpp: rbox_open_mailbox at line 374 Debug: [START] rbox-storage.cpp: rbox_read_header at line 281 Debug: [END] rbox-storage.cpp: rbox_read_header at line 313 Debug: [END] rbox-storage.cpp: rbox_mailbox_open at line 679 Debug: [START] rbox-save.cpp: rbox_save_alloc at line 57 Debug: [END] rbox-save.cpp: rbox_save_alloc at line 78 Debug: [START] rbox-mail.cpp: rbox_mail_alloc at line 100 Debug: [END] rbox-mail.cpp: rbox_mail_alloc at line 114 Debug: [START] rbox-storage.cpp: rbox_mailbox_get_metadata at line 851 Debug: [END] rbox-storage.cpp: rbox_mailbox_get_metadata at line 869 Debug: [START] rbox-copy.cpp: rbox_mail_copy at line 42 Debug: [START] rbox-copy.cpp: rbox_mail_storage_copy at line 288 Debug: [START] rbox-storage.cpp: rbox_open_rados_connection at line 432 Debug: [START] rbox-storage.cpp: read_plugin_configuration at line 402 Debug: reading plugin conf: rados_check_empty_mailboxes=false Debug: reading plugin conf: rados_save_log=/var/mail/rbox/save_file.log Debug: reading plugin conf: rados_user_name=client.admin Debug: reading plugin conf: rbox_bugfix_cephfs_21652=false Debug: reading plugin conf: rbox_ceph_aio_wait_for_safe_and_cb=false Debug: reading plugin conf: rbox_ceph_write_chunks=false Debug: reading plugin conf: rbox_cfg_object_name=rbox_cfg Debug: reading plugin conf: rbox_chunk_size=(null) Debug: reading plugin conf: rbox_cluster_name=ceph Debug: reading plugin conf: rbox_index_pool_name=(null) Debug: reading plugin conf: rbox_object_search_method=(null) Debug: reading plugin conf: rbox_object_search_threads=(null) Debug: reading plugin conf: rbox_pool_name=mail_storage Debug: reading plugin conf: rbox_write_method=(null) Debug: [END] rbox-storage.cpp: read_plugin_configuration at line 423 Debug: setting wait method to WAIT_FOR_COMPLETE_AND_CB Error: Exception: setting up ceph connection: std::bad_alloc Error: Open rados connection. Error(-1,Operation not permitted) (pool_name(mail_storage), cluster_name(ceph), rados_user_name(client.admin), alt_storage(0), alt_dir((null)) ) Debug: [END] rbox-copy.cpp: rbox_mail_storage_copy at line 312, ret == -1, connection to rados failed Error: ERROR, cannot open rados connection (rbox_mail_storage_copy) Debug: [END] rbox-copy.cpp: rbox_mail_copy at line 57 Debug: [START] rbox-save.cpp: rbox_transaction_save_rollback at line 839 Debug: [START] rbox-save.cpp: clean_up_failed at line 423 Debug: [START] rbox-save.cpp: clean_up_mail_object_list at line 827 Debug: [END] rbox-save.cpp: clean_up_mail_object_list at line 835 Debug: [END] rbox-save.cpp: clean_up_failed at line 449 Debug: [START] rbox-save.cpp: clean_up_mail_object_list at line 827 Debug: [END] rbox-save.cpp: clean_up_mail_object_list at line 835 Debug: [END] rbox-save.cpp: rbox_transaction_save_rollback at line 871 msgid=e86beb9e-8f57-46ba-7f7b-300cb1f4524f@mydomainfortest.com: save failed to INBOX: BUG: Unknown internal error Debug: [START] rbox-storage.cpp: rbox_mailbox_close at line 704 Debug: [END] rbox-storage.cpp: rbox_mailbox_close at line 715

From the log, I know here is the error where occurred. L453 src/storage-rbox/rbox-storage.cpp 

 ret = rados_storage->open_connection(rbox->storage->config->get_pool_name(),
                                         rbox->storage->config->get_index_pool_name(), 
                                         rbox->storage->config->get_rados_cluster_name(),
                                         rbox->storage->config->get_rados_username());
jrse commented 1 year ago

can you try with the container it worked for me. maybe you see differences with your local setup. Also please track the crash down, where does it fail in open_connection ?

jrse commented 1 year ago

did you find the reason for your problem?

cryptodriver commented 1 year ago

did you find the reason for your problem?

Not for now.

Error: Exception: setting up ceph connection: std::bad_alloc

This really, really confused me. With no further information.