search

grosjo / fts-xapian

Dovecot FTS plugin based on Xapian
GNU Lesser General Public License v2.1
93 stars 20 forks source link

Connection refused - auth-userdb #58

Closed richardshanasy closed 3 years ago

richardshanasy commented 3 years ago

Hi, I'm getting the following error: Error: auth-master: userdb list: connect(/var/run/dovecot/auth-userdb) failed: Connection refused Panic: file auth-master.c: line 439 (auth_master_unset_io): assertion failed: (conn->to == NULL)

Is that to do with this requirement: "You will need to configure properly Users Home Directories in dovecot configuration"

I'm using system users and I was afraid of losing every users mail...

Part of my dovecot config looks like this:

mail_privileged_group = mail
mail_location = mbox:~/mail:INBOX=/var/mail/%u

userdb {
  driver = passwd
}
passdb {
  driver = pam
}

protocol imap {
  mail_plugins = $mail_plugins fts fts_xapian
}

......
   fts = xapian
   fts_xapian = partial=3 full=20 attachments=0 verbose=2
   fts_autoindex_exclude = \Trash
   fts_enforced = yes
   fts_autoindex = yes

Thanks so much for all your help!!

grosjo commented 3 years ago

Can you copy/paste the output of "doveconf -n" ?

richardshanasy commented 3 years ago

Thanks so much for your help, See below. I added the mail_home setting and now it has a different error. I think the issue might be because they're system users vs virtual. 

root@mail:/home/richo# dovecot -n
# 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 ()
# OS: Linux 5.4.0-48-generic x86_64 Ubuntu 20.04.1 LTS
# Hostname: mail
disable_plaintext_auth = no
login_greeting = Shanasy Mail ready.
mail_home = /home/%n
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace inbox {
  inbox = yes
  location =
  mailbox "All Mail" {
    auto = subscribe
    special_use = \All
  }
  mailbox Archive {
    comment = Archived Mail
    special_use = \Archive
  }
  mailbox Drafts {
    auto = no
    special_use = \Drafts
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox Spam {
    auto = create
    special_use = \Junk
  }
  mailbox Trash {
    auto = no
    special_use = \Trash
  }
  mailbox virtual/Flagged {
    comment = All my flagged messages
    special_use = \Flagged
  }
  prefix =
}
passdb {
  driver = pam
}
plugin {
  fts = xapian
  fts_autoindex = yes
  fts_autoindex_exclude = \Trash
  fts_enforced = yes
  fts_xapian = partial=3 full=20 attachments=0 verbose=2
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_default = /var/lib/dovecot/sieve/default.sieve
  sieve_global = /var/lib/dovecot/sieve/global/
}
protocols = imap lmtp
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
}
ssl = required
ssl_cert = </etc/letsencrypt/live/mail.shanasy.com/fullchain.pem
ssl_key = # hidden, use -P to show it
userdb {
  driver = passwd
}
protocol lmtp {
  mail_plugins = " sieve"
  postmaster_address = postmaster@shanasy.com
}
protocol lda {
  mail_plugins = " sieve"
}
protocol imap {
  mail_plugins = " fts fts_xapian"
}
home/richo# doveadm index -A -q \*
doveadm(nobody): Error: mkdir(/nonexistent/mail/.imap) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
doveadm(systemd-coredump): Error: mkdir(//mail/.imap) failed: Permission denied (euid=999(systemd-coredump) egid=999(systemd-coredump) missing +w perm: /, dir owned by 0:0 mode=0755)
doveadm(lxd): Error: mkdir(/var/snap/lxd/common/lxd/mail/.imap) failed: Permission denied (euid=998(lxd) egid=100(users) missing +w perm: /var/snap/lxd/common/lxd, dir owned by 0:0 mode=0711)
doveadm(nobody): Error: mkdir(/nonexistent/mail/.imap) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
grosjo commented 3 years ago

Hello

I am a bit puzzled by these lines

mail_home = /home/%n mail_location = mbox:~/mail:INBOX=/var/mail/%u

Can you try removing the "mail_home" entry ?

For the plugin part, I have

mail_plugins = fts_xapian fts

in the beginning of my conf file

and in the plugin part, I added

plugin { plugin = fts_xapian managesieve sieve

fts = xapian

...

Finally, can you send me more output of the log file ?

slavkoja commented 3 years ago

IMO the problem seems to be PAM auth DB, in combination with -A it looks, as system accounts are checked too and some of them have no HOME (mkdir(//mail/.imap)) or have set HOME to /nonexistent. And for me it sounds as no FTS/Xapian mistake.

I suggest inspection, what user DB returns in details, especially the HOME/MAIL paths. If possible, try limit PAM users by first_valid_uid/last_valid_uid (or their GID brothers).

richardshanasy commented 3 years ago

I updated the settings:

root@mail:/home/richo# dovecot -n
# 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 ()
# OS: Linux 5.4.0-48-generic x86_64 Ubuntu 20.04.1 LTS
# Hostname: mail
disable_plaintext_auth = no
login_greeting = Shanasy Mail ready.
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_plugins = fts_xapian fts
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace inbox {
  inbox = yes
  location =
  mailbox "All Mail" {
    auto = subscribe
    special_use = \All
  }
  mailbox Archive {
    comment = Archived Mail
    special_use = \Archive
  }
  mailbox Drafts {
    auto = no
    special_use = \Drafts
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox Spam {
    auto = create
    special_use = \Junk
  }
  mailbox Trash {
    auto = no
    special_use = \Trash
  }
  mailbox virtual/Flagged {
    comment = All my flagged messages
    special_use = \Flagged
  }
  prefix =
}
passdb {
  driver = pam
}
plugin {
  fts = xapian
  fts_autoindex = yes
  fts_autoindex_exclude = \Trash
  fts_enforced = yes
  fts_xapian = partial=3 full=20 attachments=0 verbose=2
  plugin = fts_xapian managesieve sieve
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_default = /var/lib/dovecot/sieve/default.sieve
  sieve_global = /var/lib/dovecot/sieve/global/
}
protocols = imap lmtp
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
}
ssl = required
ssl_cert = </etc/letsencrypt/live/mail.shanasy.com/fullchain.pem
ssl_key = # hidden, use -P to show it
userdb {
  driver = passwd
}
protocol lmtp {
  mail_plugins = fts_xapian fts sieve
  postmaster_address = postmaster@shanasy.com
}
protocol lda {
  mail_plugins = fts_xapian fts sieve
}
protocol imap {
  mail_plugins = fts_xapian fts fts fts_xapian

root@mail:/home/richo# doveadm index -A -q \*
doveadm(nobody): Error: mkdir(/nonexistent/mail/.imap/xapian-indexes) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
doveadm(nobody): Error: FTS Xapian: can not create '/nonexistent/mail/.imap/xapian-indexes'
doveadm(nobody): Error: fts: Failed to initialize backend 'xapian': /
doveadm(nobody): Error: mkdir(/nonexistent/mail/.imap) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
doveadm(systemd-coredump): Error: mkdir(//mail/.imap/xapian-indexes) failed: Permission denied (euid=999(systemd-coredump) egid=999(systemd-coredump) missing +w perm: /, dir owned by 0:0 mode=0755)
doveadm(systemd-coredump): Error: FTS Xapian: can not create '//mail/.imap/xapian-indexes'
doveadm(systemd-coredump): Error: fts: Failed to initialize backend 'xapian': /
doveadm(systemd-coredump): Error: mkdir(//mail/.imap) failed: Permission denied (euid=999(systemd-coredump) egid=999(systemd-coredump) missing +w perm: /, dir owned by 0:0 mode=0755)
doveadm(richo): Info: FTS Xapian: Starting with partial=3 full=20 attachments=0 verbose=2
doveadm(richo): Info: FTS Xapian: fts_backend_xapian_deinit
doveadm(richo): Info: Deinit /home/richo/mail/.imap/xapian-indexes
doveadm(richo): Info: FTS Xapian: Unset box '(null)' ((null))
doveadm(richo): Info: FTS Xapian: Committed 'unset_box' in 0 ms
doveadm(lxd): Error: mkdir(/var/snap/lxd/common/lxd/mail/.imap/xapian-indexes) failed: Permission denied (euid=998(lxd) egid=100(users) missing +w perm: /var/snap/lxd/common/lxd, dir owned by 0:0 mode=0711)
doveadm(lxd): Error: FTS Xapian: can not create '/var/snap/lxd/common/lxd/mail/.imap/xapian-indexes'
doveadm(lxd): Error: fts: Failed to initialize backend 'xapian': /
doveadm(lxd): Error: mkdir(/var/snap/lxd/common/lxd/mail/.imap) failed: Permission denied (euid=998(lxd) egid=100(users) missing +w perm: /var/snap/lxd/common/lxd, dir owned by 0:0 mode=0711)
doveadm(r): Info: FTS Xapian: Starting with partial=3 full=20 attachments=0 verbose=2
doveadm(r): Info: FTS Xapian: fts_backend_xapian_deinit
doveadm(r): Info: Deinit /home/r/mail/.imap/xapian-indexes
doveadm(r): Info: FTS Xapian: Unset box '(null)' ((null))
doveadm(r): Info: FTS Xapian: Committed 'unset_box' in 0 ms
doveadm(b): Info: FTS Xapian: Starting with partial=3 full=20 attachments=0 verbose=2
doveadm(b): Info: FTS Xapian: fts_backend_xapian_deinit
doveadm(b): Info: Deinit /home/b/mail/.imap/xapian-indexes
doveadm(b): Info: FTS Xapian: Unset box '(null)' ((null))
doveadm(b): Info: FTS Xapian: Committed 'unset_box' in 0 ms
doveadm(bel): Info: FTS Xapian: Starting with partial=3 full=20 attachments=0 verbose=2
doveadm(bel): Info: FTS Xapian: fts_backend_xapian_deinit
doveadm(bel): Info: Deinit /home/bel/mail/.imap/xapian-indexes
doveadm(bel): Info: FTS Xapian: Unset box '(null)' ((null))
doveadm(bel): Info: FTS Xapian: Committed 'unset_box' in 0 ms
doveadm(john): Info: FTS Xapian: Starting with partial=3 full=20 attachments=0 verbose=2
doveadm(john): Info: FTS Xapian: fts_backend_xapian_deinit
doveadm(john): Info: Deinit /home/john/mail/.imap/xapian-indexes
doveadm(john): Info: FTS Xapian: Unset box '(null)' ((null))
doveadm(john): Info: FTS Xapian: Committed 'unset_box' in 0 ms
doveadm(sue): Info: FTS Xapian: Starting with partial=3 full=20 attachments=0 verbose=2
doveadm(sue): Info: FTS Xapian: fts_backend_xapian_deinit
doveadm(sue): Info: Deinit /home/sue/mail/.imap/xapian-indexes
doveadm(sue): Info: FTS Xapian: Unset box '(null)' ((null))
doveadm(sue): Info: FTS Xapian: Committed 'unset_box' in 0 ms
doveadm(nextcloud): Info: FTS Xapian: Starting with partial=3 full=20 attachments=0 verbose=2
doveadm(nextcloud): Info: FTS Xapian: fts_backend_xapian_deinit
doveadm(nextcloud): Info: Deinit /home/nextcloud/mail/.imap/xapian-indexes
doveadm(nextcloud): Info: FTS Xapian: Unset box '(null)' ((null))
doveadm(nextcloud): Info: FTS Xapian: Committed 'unset_box' in 0 msdoveadm(nobody): Error: mkdir(/nonexistent/mail/.imap/xapian-indexes) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
doveadm(nobody): Error: FTS Xapian: can not create '/nonexistent/mail/.imap/xapian-indexes'
doveadm(nobody): Error: fts: Failed to initialize backend 'xapian': /
doveadm(nobody): Error: mkdir(/nonexistent/mail/.imap) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
root@mail:/home/richo

And sys log:

root@mail:/home/richo# tail -n 50 /var/log/syslog
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=X-Google-DKIM-Signature,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Unknown header (indexing) 'xgoogledkimsignature'
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=X-Gm-Message-State,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Unknown header (indexing) 'xgmmessagestate'
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=MIME-Version,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Unknown header (indexing) 'mimeversion'
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=X-Received,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Unknown header (indexing) 'xreceived'
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=Received,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Unknown header (indexing) 'received'
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=Date,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Unknown header (indexing) 'date'
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=Message-ID,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_build_more
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Query= uid:"3928"
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Ngram(XMID) -> 5 items (total 0 KB)
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_unset_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=Subject,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_build_more
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Query= uid:"3928"
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Ngram(S) -> 33 items (total 0 KB)
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_unset_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=From,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_build_more
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Query= uid:"3928"
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Ngram(A) -> 91 items (total 0 KB)
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_unset_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=To,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_build_more
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Query= uid:"3928"
Oct 11 23:30:08 mail dovecot: indexer-worker(b)<95870><sHFADFv6gl9+dgEAZU03Dg>: FTS Xapian: NGRAM(body,XBDY) -> 571 items, max length=15, (total 3 KB)
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Ngram(XTO) -> 91 items (total 0 KB)
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_unset_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=Content-Type,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Unknown header (indexing) 'contenttype'
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_set_build_key
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: New part (Header=Bcc,Type=(null),Disposition=(null))
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: fts_backend_xapian_update_build_more
Oct 11 23:30:08 mail dovecot: indexer-worker(sue)<95873><6EFmEHb5gl+BdgEAZU03Dg>: FTS Xapian: Query= uid:"3928"
root@mail:/home/richo#
richardshanasy commented 3 years ago

Some more logs. Sorry about the amount... Thanks so much for your help. 


`dovecot.service - Dovecot IMAP/POP3 email server
     Loaded: loaded (/lib/systemd/system/dovecot.service; enabled;>     Active: active (running) since Sun 2020-10-11 23:33:27 AEDT; >       Docs: man:dovecot(1)
             http://wiki2.dovecot.org/
   Main PID: 95960 (dovecot)
      Tasks: 15 (limit: 9220)
     Memory: 504.6M
     CGroup: /system.slice/dovecot.service
             ├─95960 /usr/sbin/dovecot -F
             ├─95973 dovecot/anvil
             ├─95974 dovecot/log
             ├─95975 dovecot/config
             ├─95980 dovecot/stats
             ├─95981 dovecot/auth
             ├─95983 dovecot/indexer
             ├─95984 dovecot/indexer-worker
             ├─95985 dovecot/indexer-worker
             ├─95986 dovecot/indexer-worker
             ├─95987 dovecot/indexer-worker
             ├─95990 dovecot/indexer-worker
             ├─95992 dovecot/indexer-worker
             ├─95993 dovecot/indexer-worker
             └─95994 dovecot/indexer-worker

Oct 11 23:33:42 mail dovecot[95974]: indexer-worker(b)<95990><yJu1>Oct 11 23:33:42 mail dovecot[95974]: indexer-worker(b)<95990><iNPl>Oct 11 23:33:42 mail dovecot[95974]: indexer-worker(b)<95990><6BQX>Oct 11 23:33:42 mail dovecot[95974]: indexer-worker(john)<95992><M>Oct 11 23:33:42 mail dovecot[95974]: indexer-worker(john)<95992><+>Oct 11 23:33:42 mail dovecot[95974]: indexer-worker(john)<95992><6>Oct 11 23:34:28 mail dovecot[95974]: indexer-worker: Error: termin>Oct 11 23:34:28 mail dovecot[95974]: indexer-worker: Error:   what>Oct 11 23:34:30 mail dovecot[95974]: indexer: Error: Indexer worke>Oct 11 23:34:30 mail dovecot[95974]: indexer-worker(r)<95989><kCOr>~
~
~
~
~
~
~
~
root@mail:/home/richo# tail -n 50 /var/lo
local/ lock/  log/
root@mail:/home/richo# tail -n 50 /var/log/syslog
Oct 11 23:33:42 mail dovecot: indexer-worker(systemd-coredump)<95985><NNHiC6b7gl/xdgEAZU03Dg>: Error: mkdir(//mail/.imap) failed: Permission denied (euid=999(systemd-coredump) egid=999(systemd-coredump) missing +w perm: /, dir owned by 0:0 mode=0755)
Oct 11 23:33:42 mail dovecot: indexer-worker(systemd-coredump)<95985><NNHiC6b7gl/xdgEAZU03Dg>: Error: mkdir(//mail/.imap) failed: Permission denied (euid=999(systemd-coredump) egid=999(systemd-coredump) missing +w perm: /, dir owned by 0:0 mode=0755)
Oct 11 23:33:42 mail dovecot: indexer-worker(systemd-coredump)<95985><NNHiC6b7gl/xdgEAZU03Dg>: Error: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2020-10-11 23:33:42]
Oct 11 23:33:42 mail dovecot: indexer-worker(systemd-coredump)<95985><NNHiC6b7gl/xdgEAZU03Dg>: Error: Syncing mailbox INBOX failed: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2020-10-11 23:33:42]
Oct 11 23:33:42 mail dovecot: indexer-worker(lxd)<95987><dn5+DKb7gl/zdgEAZU03Dg>: Error: mkdir(/var/snap/lxd/common/lxd/mail/.imap/xapian-indexes) failed: Permission denied (euid=998(lxd) egid=100(users) missing +w perm: /var/snap/lxd/common/lxd, dir owned by 0:0 mode=0711)
Oct 11 23:33:42 mail dovecot: indexer-worker(lxd)<95987><dn5+DKb7gl/zdgEAZU03Dg>: Error: FTS Xapian: can not create '/var/snap/lxd/common/lxd/mail/.imap/xapian-indexes'
Oct 11 23:33:42 mail dovecot: indexer-worker(lxd)<95987><dn5+DKb7gl/zdgEAZU03Dg>: Error: fts: Failed to initialize backend 'xapian': /Oct 11 23:33:42 mail dovecot: indexer-worker(lxd)<95987><dn5+DKb7gl/zdgEAZU03Dg>: Error: mkdir(/var/snap/lxd/common/lxd/mail/.imap) failed: Permission denied (euid=998(lxd) egid=100(users) missing +w perm: /var/snap/lxd/common/lxd, dir owned by 0:0 mode=0711)
Oct 11 23:33:42 mail dovecot: indexer-worker(lxd)<95987><dn5+DKb7gl/zdgEAZU03Dg>: Error: mkdir(/var/snap/lxd/common/lxd/mail/.imap) failed: Permission denied (euid=998(lxd) egid=100(users) missing +w perm: /var/snap/lxd/common/lxd, dir owned by 0:0 mode=0711)
Oct 11 23:33:42 mail dovecot: indexer-worker(lxd)<95987><dn5+DKb7gl/zdgEAZU03Dg>: Error: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2020-10-11 23:33:42]
Oct 11 23:33:42 mail dovecot: indexer-worker(lxd)<95987><dn5+DKb7gl/zdgEAZU03Dg>: Error: Syncing mailbox INBOX failed: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2020-10-11 23:33:42]
Oct 11 23:33:42 mail dovecot: indexer-worker(richo)<95986><Da8tDKb7gl/ydgEAZU03Dg>: Indexed 0 messages in INBOX
Oct 11 23:33:42 mail dovecot: indexer-worker(bel)<95984><eLl7DKb7gl/wdgEAZU03Dg>: Indexed 0 messages in INBOX
Oct 11 23:33:42 mail dovecot: indexer-worker(nobody)<95994><vioIDqb7gl/6dgEAZU03Dg>: Error: mkdir(/nonexistent/mail/.imap/xapian-indexes) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
Oct 11 23:33:42 mail dovecot: indexer-worker(nobody)<95994><vioIDqb7gl/6dgEAZU03Dg>: Error: FTS Xapian: can not create '/nonexistent/mail/.imap/xapian-indexes'
Oct 11 23:33:42 mail dovecot: indexer-worker(nobody)<95994><vioIDqb7gl/6dgEAZU03Dg>: Error: fts: Failed to initialize backend 'xapian': /
Oct 11 23:33:42 mail dovecot: indexer-worker(nobody)<95994><vioIDqb7gl/6dgEAZU03Dg>: Error: mkdir(/nonexistent/mail/.imap) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
Oct 11 23:33:42 mail dovecot: indexer-worker(nobody)<95994><vioIDqb7gl/6dgEAZU03Dg>: Error: mkdir(/nonexistent/mail/.imap) failed: Permission denied (euid=65534(nobody) egid=65534(nogroup))
Oct 11 23:33:42 mail dovecot: indexer-worker(nobody)<95994><vioIDqb7gl/6dgEAZU03Dg>: Error: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2020-10-11 23:33:42]
Oct 11 23:33:42 mail dovecot: indexer-worker(nobody)<95994><vioIDqb7gl/6dgEAZU03Dg>: Error: Syncing mailbox INBOX failed: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2020-10-11 23:33:42]
Oct 11 23:33:42 mail dovecot: indexer-worker(nextcloud)<95985><mIIjDab7gl/xdgEAZU03Dg>: Indexed 0 messages in INBOX
Oct 11 23:33:42 mail dovecot: indexer-worker(sue)<95993><Snm7Dab7gl/5dgEAZU03Dg>: Indexed 0 messages in Starred
Oct 11 23:33:42 mail dovecot: indexer-worker(r)<95989><kg7JDKb7gl/1dgEAZU03Dg>: Indexed 0 messages in Starred
Oct 11 23:33:42 mail dovecot: indexer-worker(john)<95992></41lDab7gl/4dgEAZU03Dg>: Indexed 0 messages in Deleted Messages
Oct 11 23:33:42 mail dovecot: indexer-worker(b)<95990><cJolDab7gl/2dgEAZU03Dg>: Indexed 0 messages in Trash
Oct 11 23:33:42 mail dovecot: indexer-worker(john)<95992><OLsZEab7gl/4dgEAZU03Dg>: Indexed 0 messages in Starred
Oct 11 23:33:42 mail dovecot: indexer-worker(john)<95992><YNbBEab7gl/4dgEAZU03Dg>: Indexed 0 messages in Spam
Oct 11 23:33:42 mail dovecot: indexer-worker(b)<95990><iCx7Eab7gl/2dgEAZU03Dg>: Indexed 0 messages in job apps
Oct 11 23:33:42 mail dovecot: indexer-worker(b)<95990><yJu1E6b7gl/2dgEAZU03Dg>: Indexed 0 messages in info@beachsidedoctors.com.au
Oct 11 23:33:42 mail dovecot: indexer-worker(b)<95990><iNPlE6b7gl/2dgEAZU03Dg>: Indexed 0 messages in bronwyn@beachsidedoctors.com.au
Oct 11 23:33:42 mail dovecot: indexer-worker(b)<95990><6BQXFKb7gl/2dgEAZU03Dg>: Indexed 0 messages in Starred
Oct 11 23:33:42 mail dovecot: indexer-worker(john)<95992><MJ0uE6b7gl/4dgEAZU03Dg>: Indexed 0 messages in Important
Oct 11 23:33:42 mail dovecot: indexer-worker(john)<95992><+P04F6b7gl/4dgEAZU03Dg>: Indexed 0 messages in Drafts
Oct 11 23:33:42 mail dovecot: indexer-worker(john)<95992><6CFrF6b7gl/4dgEAZU03Dg>: Indexed 0 messages in Notes
Oct 11 23:34:28 mail dovecot: indexer-worker: Error: terminate called after throwing an instance of 'std::bad_alloc'
Oct 11 23:34:28 mail dovecot: indexer-worker: Error:   what():  std::bad_alloc
Oct 11 23:34:30 mail dovecot: indexer: Error: Indexer worker disconnected, discarding 7 requests for r
Oct 11 23:34:30 mail dovecot: indexer-worker(r)<95989><kCOrD6b7gl/1dgEAZU03Dg>: Fatal: master: service(indexer-worker): child 95989 killed with signal 6 (core dumped)
Oct 11 23:34:54 mail dovecot: indexer-worker(john)<95992><YFmdF6b7gl/4dgEAZU03Dg>: Indexed 403 messages in All Mail (UIDs 872..1274)
Oct 11 23:35:01 mail CRON[96009]: (www-data) CMD (php -f /var/www/html/cron.php)
Oct 11 23:35:11 mail dovecot: imap-login: Login: user=<r>, method=PLAIN, rip=192.168.1.1, lip=192.168.1.200, mpid=96020, TLS, session=<oPenaWSxUovAqAEB>
Oct 11 23:35:20 mail dovecot: imap(r)<96020><oPenaWSxUovAqAEB>: Connection closed (UID FETCH finished 6.367 secs ago) in=5012 out=39285 deleted=0 expunged=0 trashed=0 hdr_count=10 hdr_bytes=5886 body_count=1 body_bytes=8885
Oct 11 23:35:23 mail dovecot: indexer-worker(sue)<95993><IGweD6b7gl/5dgEAZU03Dg>: Indexed 676 messages in Important (UIDs 5757..6432)
Oct 11 23:35:23 mail dovecot: indexer-worker(sue)<95993><CFsZBgv8gl/5dgEAZU03Dg>: Indexed 4 messages in Drafts (UIDs 317..320)
Oct 11 23:35:23 mail dovecot: indexer-worker(sue)<95993><6D4oEAv8gl/5dgEAZU03Dg>: Indexed 0 messages in Personal
Oct 11 23:35:23 mail dovecot: indexer-worker(sue)<95993><mINnEwv8gl/5dgEAZU03Dg>: Indexed 0 messages in Notes
Oct 11 23:35:23 mail dovecot: indexer-worker(sue)<95993><yFMsFwv8gl/5dgEAZU03Dg>: Indexed 0 messages in Medical
Oct 11 23:35:23 mail dovecot: indexer-worker(sue)<95993><ONEPGQv8gl/5dgEAZU03Dg>: Indexed 0 messages in Gardening
Oct 11 23:35:24 mail dovecot: indexer-worker(sue)<95993><gCFIGwv8gl/5dgEAZU03Dg>: Indexed 5 messages in Art (UIDs 1..5)
Oct 11 23:35:54 mail dovecot: indexer-worker(john)<95992><KMg0Eu77gl/4dgEAZU03Dg>: Indexed 1101 messages in Sent (UIDs 1..1101)
root@mail:/home/richo
```#`
richardshanasy commented 3 years ago

Thanks @slavkoja. I might try indexing one real user at a time and see what happens

grosjo commented 3 years ago

@richardshanasy you can also force the home by setting: mail_location = mbox:/var/mail/℅u (provided /var/mail does exist)

richardshanasy commented 3 years ago

Thanks so much guys. Seems to be indexing okay now.