Closed LuftschloesserUser closed 2 years ago
With which version of dovecot are you compiling ?
Can you test with latest git ?
`make all-recursive make[1]: Entering directory '/root/fts-xapian' Making all in src make[2]: Entering directory '/root/fts-xapian/src' /bin/bash ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c -o fts-xapian-plugin.lo fts-xapian-plugin.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -fPIC -DPIC -o .libs/fts-xapian-plugin.o In file included from fts-xapian-plugin.c:4:0: fts-xapian-plugin.c: In function ‘fts_xapian_mail_user_deinit’: fts-xapian-plugin.h:52:46: warning: implicit declaration of function ‘MODULE_CONTEXT_REQUIRE’; did you mean ‘MODULE_CONTEXT_DEFINE’? [-Wimplicit-function-declaration]
^
fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’
struct fts_xapian_user *fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user);
^~~~~~~
fts-xapian-plugin.h:52:46: warning: initialization makes pointer from integer without a cast [-Wint-conversion]
^
fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’
struct fts_xapian_user fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user);
^~~~~~~
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -o fts-xapian-plugin.o >/dev/null 2>&1
mv -f .deps/fts-xapian-plugin.Tpo .deps/fts-xapian-plugin.Plo
/bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c -o fts-backend-xapian.lo fts-backend-xapian.cpp
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -fPIC -DPIC -o .libs/fts-backend-xapian.o
fts-backend-xapian.cpp: In function ‘int fts_backend_xapian_optimize(fts_backend)’:
fts-backend-xapian.cpp:534:84: warning: too many arguments for format [-Wformat-extra-args]
i_error("FTS Xapian: Optimize (3) Can not select IDs : %s",selectUIDs,zErrMsg);
^
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -o fts-backend-xapian.o >/dev/null 2>&1
mv -f .deps/fts-backend-xapian.Tpo .deps/fts-backend-xapian.Plo
/bin/bash ../libtool --tag=CXX --mode=link g++ -std=gnu++11 -g -O2 -module -avoid-version -o lib21_fts_xapian_plugin.la -rpath /usr/lib/dovecot/modules fts-xapian-plugin.lo fts-backend-xapian.lo -L/usr/lib/x86_64-linux-gnu -lxapian -licuio -licui18n -licuuc -licudata -lsqlite3
libtool: link: g++ -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o .libs/fts-xapian-plugin.o .libs/fts-backend-xapian.o -L/usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/libxapian.so -licuio -licui18n -licuuc -licudata /usr/lib/x86_64-linux-gnu/libsqlite3.so -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o -g -O2 -Wl,-soname -Wl,lib21_fts_xapian_plugin.so -o .libs/lib21_fts_xapian_plugin.so
libtool: link: ar cru .libs/lib21_fts_xapian_plugin.a fts-xapian-plugin.o fts-backend-xapian.o
ar: u' modifier ignored since
D' is the default (see U') libtool: link: ranlib .libs/lib21_fts_xapian_plugin.a libtool: link: ( cd ".libs" && rm -f "lib21_fts_xapian_plugin.la" && ln -s "../lib21_fts_xapian_plugin.la" "lib21_fts_xapian_plugin.la" ) make[2]: Leaving directory '/root/fts-xapian/src' make[2]: Entering directory '/root/fts-xapian' make[2]: Leaving directory '/root/fts-xapian' make[1]: Leaving directory '/root/fts-xapian'
Thats the result. Thanks for your fast reply and commit!
Please try again with latest git
`make all-recursive make[1]: Entering directory '/root/fts-xapian' Making all in src make[2]: Entering directory '/root/fts-xapian/src' /bin/bash ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c -o fts-xapian-plugin.lo fts-xapian-plugin.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -fPIC -DPIC -o .libs/fts-xapian-plugin.o In file included from fts-xapian-plugin.c:4:0: fts-xapian-plugin.c: In function ‘fts_xapian_mail_user_deinit’: fts-xapian-plugin.h:52:46: warning: implicit declaration of function ‘MODULE_CONTEXT_REQUIRE’; did you mean ‘MODULE_CONTEXT_DEFINE’? [-Wimplicit-function-declaration]
^
fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’
struct fts_xapian_user *fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user);
^~~~~~~
fts-xapian-plugin.h:52:46: warning: initialization makes pointer from integer without a cast [-Wint-conversion]
^
fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’
struct fts_xapian_user *fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user);
^~~~~~~
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -o fts-xapian-plugin.o >/dev/null 2>&1
mv -f .deps/fts-xapian-plugin.Tpo .deps/fts-xapian-plugin.Plo
/bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c -o fts-backend-xapian.lo fts-backend-xapian.cpp
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -fPIC -DPIC -o .libs/fts-backend-xapian.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -o fts-backend-xapian.o >/dev/null 2>&1
mv -f .deps/fts-backend-xapian.Tpo .deps/fts-backend-xapian.Plo
/bin/bash ../libtool --tag=CXX --mode=link g++ -std=gnu++11 -g -O2 -module -avoid-version -o lib21_fts_xapian_plugin.la -rpath /usr/lib/dovecot/modules fts-xapian-plugin.lo fts-backend-xapian.lo -L/usr/lib/x86_64-linux-gnu -lxapian -licuio -licui18n -licuuc -licudata -lsqlite3
libtool: link: g++ -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o .libs/fts-xapian-plugin.o .libs/fts-backend-xapian.o -L/usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/libxapian.so -licuio -licui18n -licuuc -licudata /usr/lib/x86_64-linux-gnu/libsqlite3.so -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o -g -O2 -Wl,-soname -Wl,lib21_fts_xapian_plugin.so -o .libs/lib21_fts_xapian_plugin.so
libtool: link: ar cru .libs/lib21_fts_xapian_plugin.a fts-xapian-plugin.o fts-backend-xapian.o
ar: u' modifier ignored since
D' is the default (see U') libtool: link: ranlib .libs/lib21_fts_xapian_plugin.a libtool: link: ( cd ".libs" && rm -f "lib21_fts_xapian_plugin.la" && ln -s "../lib21_fts_xapian_plugin.la" "lib21_fts_xapian_plugin.la" ) make[2]: Leaving directory '/root/fts-xapian/src' make[2]: Entering directory '/root/fts-xapian' make[2]: Leaving directory '/root/fts-xapian' make[1]: Leaving directory '/root/fts-xapian'
So, just warnings, right ?
right, yes! You think i should try it?
so I installed it.
But it doesn't seem to be right:
doveadm fts optimize -u username Fatal: Couldn't load required plugin /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so: dlopen() failed: /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so: undefined symbol: MODULE_CONTEXT_REQUIRE
ls /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so
I did disable the module, because dovecot did not work any more.
What is your version of dovecot ?
dovecot --version
2.2.33.2 (d6601f4ec)
PLease try again latest git with your dovecot 2.2
Installation works and dovecot continues to work, but search does not work.
BTW this brings no output at all:
doveadm fts optimize -u usename
if you put "verbose=2" in your dovecot.conf and relaunch the command, what do you get ?
doveadm fts optimize -u username doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 84: Unknown setting: verbose
I tried in different locations but verbose = 2 does not seem to be a working configuration
Hello
Shall be :
plugin { fts = xapian fts_xapian = partial=3 full=20 verbose=2
fts_autoindex = yes
fts_enforced = yes
So this worked and search seems to work now too. Behaviour was a bit erratic, but maybe that was just me.
this is the output with verbose=2:
`doveadm fts optimize -u username
doveadm(username): Warning: FTS Xapian: fts_languages setting is missing doveadm(username): Info: FTS Xapian: Index path = /home/username/mail/.imap/xapian-indexes doveadm(username): Warning: FTS Xapian: RLIM AS =18014398509481984 doveadm(username): Warning: FTS Xapian: RLIM DATA =18014398509481984 doveadm(username): Warning: FTS Xapian: Free memory 3564 MB doveadm(username): Info: FTS Xapian: Starting with partial=3 full=20 verbose=2 lowmemory=250 MB vs freemem=3564 MB doveadm(username): Info: FTS Xapian: fts_backend_xapian_optimize '/home/username/mail/.imap/xapian-indexes' doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_261f3d3a4ebadd61942a000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_261f3d3a4ebadd61942a000010f3be3c) doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_d0b5310a9ab9dd61db28000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_d0b5310a9ab9dd61db28000010f3be3c) doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_0ef5462dd8cae261b108000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_0ef5462dd8cae261b108000010f3be3c) doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_add86f3a0a8e8261a258000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_add86f3a0a8e8261a258000010f3be3c) doveadm(username): Info: FTS Xapian: Deinit /home/username/mail/.imap/xapian-indexes) `
Great
So, shall be fine now
Thank you for posting the initial issue
Feel free to reopen if further problems appear
Hi Joan,
my make fails with this:
make all-recursive make[1]: Entering directory '/root/fts-xapian' Making all in src make[2]: Entering directory '/root/fts-xapian/src' /bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c -o fts-backend-xapian.lo fts-backend-xapian.cpp libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -fPIC -DPIC -o .libs/fts-backend-xapian.o fts-backend-xapian.cpp: In function ‘int fts_backend_xapian_optimize_callback(void*, int, char**, char**)’: fts-backend-xapian.cpp:491:2: error: ‘array_push_back’ was not declared in this scope array_push_back(&uids->uids,&uid); ^~~~~~~~~~~~~~~ fts-backend-xapian.cpp:491:2: note: suggested alternative: ‘array__unichars’ array_push_back(&uids->uids,&uid); ^~~~~~~~~~~~~~~ array__unichars fts-backend-xapian.cpp: In function ‘int fts_backend_xapian_optimize(fts_backend*)’: fts-backend-xapian.cpp:539:84: warning: too many arguments for format [-Wformat-extra-args] i_error("FTS Xapian: Optimize (3) Can not select IDs : %s",selectUIDs,zErrMsg); ^ fts-backend-xapian.cpp:550:6: error: ‘array_foreach_elem’ was not declared in this scope array_foreach_elem(&(uids.uids), uid) ^~~~~~~~~~~~~~~~~~ fts-backend-xapian.cpp:550:6: note: suggested alternative: ‘array_foreach_idx’ array_foreach_elem(&(uids.uids), uid) ^~~~~~~~~~~~~~~~~~ array_foreach_idx Makefile:574: recipe for target 'fts-backend-xapian.lo' failed make[2]: *** [fts-backend-xapian.lo] Error 1 make[2]: Leaving directory '/root/fts-xapian/src' Makefile:492: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/root/fts-xapian' Makefile:414: recipe for target 'all' failed make: *** [all] Error 2
Its an Ubuntu 20.04 and I had an old version of fts-xapian running fine. For the update to your new release I did a clean git clone.
Any suggestions?