grosjo / fts-xapian

Dovecot FTS plugin based on Xapian
GNU Lesser General Public License v2.1
97 stars 21 forks source link

Doesn't search for all folders #41

Closed igorzigorz closed 4 years ago

igorzigorz commented 4 years ago

dovecot 2.3.10 fts xapian 1.2.11a and roundcube 1.4.3 Several folders about 20Gb and 18000 messages. Searching text for current folder or current with subfolders - no problem. But not searching for all folders via web or doveadm. "doveadm search" in the loop for specific folder names is Ok. But "doveadm search mailbox * body sometext" or "doveadm search body sometext" outputs almost all messages.

grosjo commented 4 years ago

This is a dovecot design. fts-xapian is just the backffice. You may post your issue on the dovecot mailing list

igorzigorz commented 4 years ago

But why does fts-lucene have a different behavior? Works as expected.

grosjo commented 4 years ago

Can you share your dovecot conf ?

igorzigorz commented 4 years ago

doveconf -n

lucene.txt xapian.txt

grosjo commented 4 years ago

can you try issuing the same commands from /tmp (to avoid conflicting "*" with local folders) ?

igorzigorz commented 4 years ago

doveadm search -u user mailbox '*' body 'sometext' doveadm search -u user body 'sometext' In any place the same result. This is when fts-lucene is enabled.

grosjo commented 4 years ago

Please try with latest git

igorzigorz commented 4 years ago

Now the search is the same with or without a mailbox. But different results if the query "text" or "body". Looking for a unique string. doveadm search -u user body 'sometext' is correct, one message. But doveadm search -u user text 'sometext' finds multiple messages.

grosjo commented 4 years ago

"body" search in the message body "text" search in the whole fields (including title, from, to, ..)

igorzigorz commented 4 years ago

fts-lucene again works as expected. This is a really unique string for all messages.

igorzigorz commented 4 years ago

Ok, I'm trying to find a string that isn't exactly there (sha384 /etc/resolv.conf). doveadm search -u user text 'thisstring' found 31 messages.

grosjo commented 4 years ago

Can you clarify what exactly is "as exptected" and what is wrong currently, with an example ? (output of the commands)

igorzigorz commented 4 years ago

"as exptected" - relevant to the grep search. If I have a single message with some unique string then the search returns only one message. Not important "text" or " body". And if there is no such string, then nothing.

grosjo commented 4 years ago

Ok So what is the actual issue with fts-xapian ?

igorzigorz commented 4 years ago

Search seems fine now. In general, there was a problem with roundcube and moving a large number of messages and renaming folders. Before that, moving worked, but with error messages. Due to starting indexing. Now indexing does not start at all. Investigating the problem.

grosjo commented 4 years ago

I tested renaming folders with roundcube, the index is kept (no re-indexing) Moving large number of items is creating timeouts in roundcube in deed, but the architecture shall be dsicussed with dovecot. Let me know if something else seems broken

igorzigorz commented 4 years ago

Rechecked, all seem ok now. This issue can be closed.

grosjo commented 4 years ago

ok cool