grosjo / fts-xapian

Dovecot FTS plugin based on Xapian
GNU Lesser General Public License v2.1
91 stars 19 forks source link

fts_backend_xapian_update_expung and dovecot hang #113

Closed lovedownload closed 1 year ago

lovedownload commented 2 years ago

fts_backend_xapian_update_expunge action occurs when dovecot bulk deletes mail. I then access dovecot with the standard imap protocol and there is no response. (dovecot hang) dovecot is transaction-locked. Is there any way to solve this problem?

The log remains like this (Below is part of the log): Info: FTS Xapian: Flagging UID=49931 for expunge Info: FTS Xapian: fts_backend_xapian_update_expunge Info: FTS Xapian: Flagging UID=49932 for expunge Info: FTS Xapian: fts_backend_xapian_update_set_mailbox Info: FTS Xapian: Unset box 'test' (b0b30429f9e75b610c550000e89a100d) Info: FTS Xapian: fts_backend_xapian_release (unset_box) Info: FTS Xapian: Committed 'unset_box' in 3 ms (49932 docs in index) Info: FTS Xapian: fts_backend_xapian_update_deinit (/dovecot/xapian-indexes) Info: FTS Xapian: fts_backend_xapian_release (update_deinit) Info: FTS Xapian: Committed 'update_deinit' in 0 ms Warning: Transaction log file /dovecot/mailboxes/test/dbox-Mails/dovecot.index.log was locked for 61 seconds (rotating while syncing) Warning: Transaction log file /dovecot/storage/dovecot.map.index.log was locked for 61 seconds (mdbox syncing with expunges) Info: FTS Xapian: Deinit /dovecot/xapian-indexes)

used vesrion dovecot - 2.3.17 fts-xapian - 1.5.1

fts option fts_enforced=no fts_autoindex=yes fts_languages = en fts_tokenizers = generic email-address fts = xapian fts_xapian = partial=2 full=20 attachments=0 verbose=1 fts_autoindex_exclude = \Trash fts_autoindex_exclude2 = \Junk fts_decoder = decode2text

grosjo commented 2 years ago

@lovedownload please kindly test again with latest git

lovedownload commented 2 years ago

Tested with the latest version. All mailboxes are not delayed as in previous versions.(v1.5.1) However, only mailboxes that are in the process of being deleted are still delayed in response.

For example (v1.5.1 - Delete bulk mail from mailbox TEST1) INBOX -> RESPONSE HANG (imap fetch, list, status ...) SENT -> RESPONSE HANG (imap fetch, list, status ...) TEST1 -> RESPONSE HANG (imap fetch, list, status ...) TEST2 -> RESPONSE HANG (imap fetch, list, status ...)

For example (Last Version - Delete bulk mail from mailbox TEST1) INBOX -> RESPONSE OK SENT -> RESPONSE OK TEST1 -> RESPONSE HANG (imap fetch, list, status ...) TEST2 -> RESPONSE OK

grosjo commented 2 years ago

How many emails are you deleting in bulk on a single mailbox ?

lovedownload commented 2 years ago

In the test, 100,000 emails were deleted.

grosjo commented 2 years ago

The plugin is done as a simple replacement of fts-squat. 100k emails seems like a lot. Probably out of scope of the plugin

grosjo commented 1 year ago

Closed due to inactivity