search

grosjo / fts-xapian

Dovecot FTS plugin based on Xapian
GNU Lesser General Public License v2.1
91 stars 19 forks source link

make fails fts-backend-xapian.cpp #117

Closed LuftschloesserUser closed 2 years ago

LuftschloesserUser commented 2 years ago

Hi Joan,

my make fails with this:

make all-recursive make[1]: Entering directory '/root/fts-xapian' Making all in src make[2]: Entering directory '/root/fts-xapian/src' /bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c -o fts-backend-xapian.lo fts-backend-xapian.cpp libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -fPIC -DPIC -o .libs/fts-backend-xapian.o fts-backend-xapian.cpp: In function ‘int fts_backend_xapian_optimize_callback(void*, int, char**, char**)’: fts-backend-xapian.cpp:491:2: error: ‘array_push_back’ was not declared in this scope array_push_back(&uids->uids,&uid); ^~~~~~~~~~~~~~~ fts-backend-xapian.cpp:491:2: note: suggested alternative: ‘array__unichars’ array_push_back(&uids->uids,&uid); ^~~~~~~~~~~~~~~ array__unichars fts-backend-xapian.cpp: In function ‘int fts_backend_xapian_optimize(fts_backend*)’: fts-backend-xapian.cpp:539:84: warning: too many arguments for format [-Wformat-extra-args] i_error("FTS Xapian: Optimize (3) Can not select IDs : %s",selectUIDs,zErrMsg); ^ fts-backend-xapian.cpp:550:6: error: ‘array_foreach_elem’ was not declared in this scope array_foreach_elem(&(uids.uids), uid) ^~~~~~~~~~~~~~~~~~ fts-backend-xapian.cpp:550:6: note: suggested alternative: ‘array_foreach_idx’ array_foreach_elem(&(uids.uids), uid) ^~~~~~~~~~~~~~~~~~ array_foreach_idx Makefile:574: recipe for target 'fts-backend-xapian.lo' failed make[2]: *** [fts-backend-xapian.lo] Error 1 make[2]: Leaving directory '/root/fts-xapian/src' Makefile:492: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/root/fts-xapian' Makefile:414: recipe for target 'all' failed make: *** [all] Error 2

Its an Ubuntu 20.04 and I had an old version of fts-xapian running fine. For the update to your new release I did a clean git clone.

Any suggestions?

grosjo commented 2 years ago

With which version of dovecot are you compiling ?

grosjo commented 2 years ago

Can you test with latest git ?

LuftschloesserUser commented 2 years ago

`make all-recursive make[1]: Entering directory '/root/fts-xapian' Making all in src make[2]: Entering directory '/root/fts-xapian/src' /bin/bash ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c -o fts-xapian-plugin.lo fts-xapian-plugin.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -fPIC -DPIC -o .libs/fts-xapian-plugin.o In file included from fts-xapian-plugin.c:4:0: fts-xapian-plugin.c: In function ‘fts_xapian_mail_user_deinit’: fts-xapian-plugin.h:52:46: warning: implicit declaration of function ‘MODULE_CONTEXT_REQUIRE’; did you mean ‘MODULE_CONTEXT_DEFINE’? [-Wimplicit-function-declaration]

define FTS_XAPIAN_USER_CONTEXT_REQUIRE(obj) MODULE_CONTEXT_REQUIRE(obj, fts_xapian_user_module)

                                          ^

fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’ struct fts_xapian_user *fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user); ^~~~~~~ fts-xapian-plugin.h:52:46: warning: initialization makes pointer from integer without a cast [-Wint-conversion]

define FTS_XAPIAN_USER_CONTEXT_REQUIRE(obj) MODULE_CONTEXT_REQUIRE(obj, fts_xapian_user_module)

                                          ^

fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’ struct fts_xapian_user fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user); ^~~~~~~ libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -o fts-xapian-plugin.o >/dev/null 2>&1 mv -f .deps/fts-xapian-plugin.Tpo .deps/fts-xapian-plugin.Plo /bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c -o fts-backend-xapian.lo fts-backend-xapian.cpp libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -fPIC -DPIC -o .libs/fts-backend-xapian.o fts-backend-xapian.cpp: In function ‘int fts_backend_xapian_optimize(fts_backend)’: fts-backend-xapian.cpp:534:84: warning: too many arguments for format [-Wformat-extra-args] i_error("FTS Xapian: Optimize (3) Can not select IDs : %s",selectUIDs,zErrMsg); ^ libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -o fts-backend-xapian.o >/dev/null 2>&1 mv -f .deps/fts-backend-xapian.Tpo .deps/fts-backend-xapian.Plo /bin/bash ../libtool --tag=CXX --mode=link g++ -std=gnu++11 -g -O2 -module -avoid-version -o lib21_fts_xapian_plugin.la -rpath /usr/lib/dovecot/modules fts-xapian-plugin.lo fts-backend-xapian.lo -L/usr/lib/x86_64-linux-gnu -lxapian -licuio -licui18n -licuuc -licudata -lsqlite3 libtool: link: g++ -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o .libs/fts-xapian-plugin.o .libs/fts-backend-xapian.o -L/usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/libxapian.so -licuio -licui18n -licuuc -licudata /usr/lib/x86_64-linux-gnu/libsqlite3.so -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o -g -O2 -Wl,-soname -Wl,lib21_fts_xapian_plugin.so -o .libs/lib21_fts_xapian_plugin.so libtool: link: ar cru .libs/lib21_fts_xapian_plugin.a fts-xapian-plugin.o fts-backend-xapian.o ar: u' modifier ignored sinceD' is the default (see U') libtool: link: ranlib .libs/lib21_fts_xapian_plugin.a libtool: link: ( cd ".libs" && rm -f "lib21_fts_xapian_plugin.la" && ln -s "../lib21_fts_xapian_plugin.la" "lib21_fts_xapian_plugin.la" ) make[2]: Leaving directory '/root/fts-xapian/src' make[2]: Entering directory '/root/fts-xapian' make[2]: Leaving directory '/root/fts-xapian' make[1]: Leaving directory '/root/fts-xapian'

Thats the result. Thanks for your fast reply and commit!

grosjo commented 2 years ago

Please try again with latest git

LuftschloesserUser commented 2 years ago

`make all-recursive make[1]: Entering directory '/root/fts-xapian' Making all in src make[2]: Entering directory '/root/fts-xapian/src' /bin/bash ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c -o fts-xapian-plugin.lo fts-xapian-plugin.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -fPIC -DPIC -o .libs/fts-xapian-plugin.o In file included from fts-xapian-plugin.c:4:0: fts-xapian-plugin.c: In function ‘fts_xapian_mail_user_deinit’: fts-xapian-plugin.h:52:46: warning: implicit declaration of function ‘MODULE_CONTEXT_REQUIRE’; did you mean ‘MODULE_CONTEXT_DEFINE’? [-Wimplicit-function-declaration]

define FTS_XAPIAN_USER_CONTEXT_REQUIRE(obj) MODULE_CONTEXT_REQUIRE(obj, fts_xapian_user_module)

                                          ^

fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’ struct fts_xapian_user *fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user); ^~~~~~~ fts-xapian-plugin.h:52:46: warning: initialization makes pointer from integer without a cast [-Wint-conversion]

define FTS_XAPIAN_USER_CONTEXT_REQUIRE(obj) MODULE_CONTEXT_REQUIRE(obj, fts_xapian_user_module)

                                          ^

fts-xapian-plugin.c:14:34: note: in expansion of macro ‘FTS_XAPIAN_USER_CONTEXT_REQUIRE’ struct fts_xapian_user *fuser = FTS_XAPIAN_USER_CONTEXT_REQUIRE(user); ^~~~~~~ libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -g -O2 -std=gnu99 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -O2 -MT fts-xapian-plugin.lo -MD -MP -MF .deps/fts-xapian-plugin.Tpo -c fts-xapian-plugin.c -o fts-xapian-plugin.o >/dev/null 2>&1 mv -f .deps/fts-xapian-plugin.Tpo .deps/fts-xapian-plugin.Plo /bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c -o fts-backend-xapian.lo fts-backend-xapian.cpp libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -fPIC -DPIC -o .libs/fts-backend-xapian.o libtool: compile: g++ -DHAVE_CONFIG_H -I. -I.. -O2 -I/usr/include/dovecot -std=gnu++11 -g -O2 -MT fts-backend-xapian.lo -MD -MP -MF .deps/fts-backend-xapian.Tpo -c fts-backend-xapian.cpp -o fts-backend-xapian.o >/dev/null 2>&1 mv -f .deps/fts-backend-xapian.Tpo .deps/fts-backend-xapian.Plo /bin/bash ../libtool --tag=CXX --mode=link g++ -std=gnu++11 -g -O2 -module -avoid-version -o lib21_fts_xapian_plugin.la -rpath /usr/lib/dovecot/modules fts-xapian-plugin.lo fts-backend-xapian.lo -L/usr/lib/x86_64-linux-gnu -lxapian -licuio -licui18n -licuuc -licudata -lsqlite3 libtool: link: g++ -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o .libs/fts-xapian-plugin.o .libs/fts-backend-xapian.o -L/usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/libxapian.so -licuio -licui18n -licuuc -licudata /usr/lib/x86_64-linux-gnu/libsqlite3.so -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o -g -O2 -Wl,-soname -Wl,lib21_fts_xapian_plugin.so -o .libs/lib21_fts_xapian_plugin.so libtool: link: ar cru .libs/lib21_fts_xapian_plugin.a fts-xapian-plugin.o fts-backend-xapian.o ar: u' modifier ignored sinceD' is the default (see U') libtool: link: ranlib .libs/lib21_fts_xapian_plugin.a libtool: link: ( cd ".libs" && rm -f "lib21_fts_xapian_plugin.la" && ln -s "../lib21_fts_xapian_plugin.la" "lib21_fts_xapian_plugin.la" ) make[2]: Leaving directory '/root/fts-xapian/src' make[2]: Entering directory '/root/fts-xapian' make[2]: Leaving directory '/root/fts-xapian' make[1]: Leaving directory '/root/fts-xapian'

grosjo commented 2 years ago

So, just warnings, right ?

LuftschloesserUser commented 2 years ago

right, yes! You think i should try it?

LuftschloesserUser commented 2 years ago

so I installed it.

But it doesn't seem to be right:

doveadm fts optimize -u username Fatal: Couldn't load required plugin /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so: dlopen() failed: /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so: undefined symbol: MODULE_CONTEXT_REQUIRE

LuftschloesserUser commented 2 years ago

ls /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so /usr/lib/dovecot/modules/lib21_fts_xapian_plugin.so

LuftschloesserUser commented 2 years ago

I did disable the module, because dovecot did not work any more.

grosjo commented 2 years ago

What is your version of dovecot ?

LuftschloesserUser commented 2 years ago

dovecot --version

2.2.33.2 (d6601f4ec)

grosjo commented 2 years ago

PLease try again latest git with your dovecot 2.2

LuftschloesserUser commented 2 years ago

Installation works and dovecot continues to work, but search does not work.

BTW this brings no output at all:

doveadm fts optimize -u usename

grosjo commented 2 years ago

if you put "verbose=2" in your dovecot.conf and relaunch the command, what do you get ?

LuftschloesserUser commented 2 years ago

doveadm fts optimize -u username doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 84: Unknown setting: verbose

I tried in different locations but verbose = 2 does not seem to be a working configuration

grosjo commented 2 years ago

Hello

Shall be :

plugin { fts = xapian fts_xapian = partial=3 full=20 verbose=2

 fts_autoindex = yes
 fts_enforced = yes
LuftschloesserUser commented 2 years ago

So this worked and search seems to work now too. Behaviour was a bit erratic, but maybe that was just me.

this is the output with verbose=2:

`doveadm fts optimize -u username

doveadm(username): Warning: FTS Xapian: fts_languages setting is missing doveadm(username): Info: FTS Xapian: Index path = /home/username/mail/.imap/xapian-indexes doveadm(username): Warning: FTS Xapian: RLIM AS =18014398509481984 doveadm(username): Warning: FTS Xapian: RLIM DATA =18014398509481984 doveadm(username): Warning: FTS Xapian: Free memory 3564 MB doveadm(username): Info: FTS Xapian: Starting with partial=3 full=20 verbose=2 lowmemory=250 MB vs freemem=3564 MB doveadm(username): Info: FTS Xapian: fts_backend_xapian_optimize '/home/username/mail/.imap/xapian-indexes' doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_261f3d3a4ebadd61942a000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_261f3d3a4ebadd61942a000010f3be3c) doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_d0b5310a9ab9dd61db28000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_d0b5310a9ab9dd61db28000010f3be3c) doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_0ef5462dd8cae261b108000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_0ef5462dd8cae261b108000010f3be3c) doveadm(username): Info: Optimize (1) /home/username/mail/.imap/xapian-indexes/db_add86f3a0a8e8261a258000010f3be3c_exp.db : Checking expunges doveadm(username): Info: Optimize (1b) Executing CREATE TABLE IF NOT EXISTS docs(ID INT PRIMARY KEY NOT NULL); doveadm(username): Info: Optimize (1c) Executing select ID from docs; doveadm(username): Info: Optimize (5a) Opening Xapian DB (/home/username/mail/.imap/xapian-indexes/db_add86f3a0a8e8261a258000010f3be3c) doveadm(username): Info: FTS Xapian: Deinit /home/username/mail/.imap/xapian-indexes) `

grosjo commented 2 years ago

Great

So, shall be fine now

Thank you for posting the initial issue

grosjo commented 2 years ago

Feel free to reopen if further problems appear