grosjo / fts-xapian

Dovecot FTS plugin based on Xapian
GNU Lesser General Public License v2.1
94 stars 20 forks source link

Segmentation fault (core dumped) if no user files exist yet #112

Closed jvolkenant closed 2 years ago

jvolkenant commented 2 years ago

I noticed that if you run doveadm index -A -q \* or doveadm fts optimize -A (likely others), without any user mailbox folders you get a Segfault. In my setup, user folders would be in the format /home/user-data/mail/mailboxes/example.com/user/.

The segfault looks to be from doveadm (per dmesg) so it may not be possible to die gracefully. Is fts-xapian able to gracefully die in this case?

root@testjammy:~/mailinabox# ls -la /home/user-data/mail/mailboxes/
total 8
drwxr-xr-x 2 mail mail 4096 Dec 21 21:52 .
drwxr-xr-x 4 root root 4096 Dec 21 21:52 ..
root@testjammy:~/mailinabox# doveadm index -A -q \*
doveadm(daemon)<22482><VCjHKjm+wmHSVwAADl9PxA>: Error: mkdir(/home/user-data/mail/mailboxes//daemon/xapian-indexes) failed: Permission denied (euid=1(daemon) egid=1(daemon) missing +w perm: /home/user-data/mail/mailboxes/, dir owned by 8:8 mode=0755)
doveadm(daemon)<22482><VCjHKjm+wmHSVwAADl9PxA>: Error: FTS Xapian: can not create '/home/user-data/mail/mailboxes//daemon/xapian-indexes'
Segmentation fault (core dumped)
dmesg:
[  338.583294] traps: doveadm[22353] general protection fault ip:7f5bb98c7b4a sp:7fffd9c006d8 error:0 in libc.so.6[7f5bb9836000+194000]
[  367.672606] traps: doveadm[22378] general protection fault ip:7fb52ed68b4a sp:7ffd96939448 error:0 in libc.so.6[7fb52ecd7000+194000]
[  368.750674] traps: doveadm[22380] general protection fault ip:7f98b9e75b4a sp:7ffceee1eb68 error:0 in libc.so.6[7f98b9de4000+194000]
[  533.253290] traps: doveadm[22448] general protection fault ip:7f8292290b4a sp:7ffd62ef06e8 error:0 in libc.so.6[7f82921ff000+194000]
[  548.182931] traps: doveadm[22458] general protection fault ip:7fd1d1df7b4a sp:7ffcde6d4b18 error:0 in libc.so.6[7fd1d1d66000+194000]
[  555.253757] traps: doveadm[22462] general protection fault ip:7fe2a63cab4a sp:7fffba59bd68 error:0 in libc.so.6[7fe2a6339000+194000]
[  591.809288] traps: doveadm[22482] general protection fault ip:7fd2bb0a0b4a sp:7ffe5a4a39d8 error:0 in libc.so.6[7fd2bb00f000+194000]
grosjo commented 2 years ago

can you share your doveconf -n ?

jvolkenant commented 2 years ago

I've reinstalled that box a few times to test install on the upcoming Ubuntu 22.04 release so I don't have that exact config and I can't exactly duplicate the same error, however if doveadm doesn't have permissions for a path you do get this error:

Using the package available from Ubuntu 22.04:

root@box:~# dpkg -l | grep xapi
ii  dovecot-fts-xapian                    1.5.2-1                                      amd64        full-text search for dovecot using xapian
ii  libxapian30:amd64                     1.4.18-4                                     amd64        Search engine library

doveconf -n

root@box:/home/user-data/mail/mailboxes# doveadm index -A -q \*
doveadm(me@jammytest.com)<70824></L11IJZszmGoFAEARG/kOA>: Info: FTS Xapian: Index path = /home/user-data/mail/mailboxes/jammytest.com/me/xapian-indexes
doveadm(me@jammytest.com)<70824></L11IJZszmGoFAEARG/kOA>: Error: stat(/home/user-data/mail/mailboxes/jammytest.com/me) failed: Permission denied (euid=8(mail) egid=8(mail) missing +x perm: /home/user-data/mail/mailboxes, dir owned by 0:0 mode=0700)
doveadm(me@jammytest.com)<70824></L11IJZszmGoFAEARG/kOA>: Error: stat(/home/user-data/mail/mailboxes/jammytest.com) failed: Permission denied (euid=8(mail) egid=8(mail) missing +x perm: /home/user-data/mail/mailboxes, dir owned by 0:0 mode=0700)
doveadm(me@jammytest.com)<70824></L11IJZszmGoFAEARG/kOA>: Error: FTS Xapian: can not create '/home/user-data/mail/mailboxes/jammytest.com/me/xapian-indexes'
Segmentation fault (core dumped)

dmesg:

[ 9933.664185] traps: doveadm[70824] general protection fault ip:7fd4bff1ab4a sp:7ffdf9187ca8 error:0 in libc.so.6[7fd4bfe89000+194000]

doveconf -n

OS: Linux 5.13.0-19-generic x86_64 Ubuntu Jammy Jellyfish (development branch) ext4
Hostname: box.jammytest.com
auth_mechanisms = plain login
default_process_limit = 1000
default_vsz_limit = 2650 M
first_valid_uid = 0
imap_idle_notify_interval = 4 mins
log_path = /var/log/mail.log
mail_access_groups = spampd
mail_location = maildir:/home/user-data/mail/mailboxes/%d/%n
mail_plugins = " fts fts_xapian"
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace inbox {
  inbox = yes
  location =
  mailbox Archive {
    auto = subscribe
    special_use = \Archive
  }
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox INBOX {
    auto = subscribe
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Spam {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  antispam_allow_append_to_spam = yes
  antispam_backend = pipe
  antispam_pipe_program = /bin/bash
  antispam_pipe_program_notspam_args = /usr/local/bin/sa-learn-pipe.sh;--ham
  antispam_pipe_program_spam_args = /usr/local/bin/sa-learn-pipe.sh;--spam
  antispam_spam_pattern_ignorecase = SPAM
  antispam_trash_pattern_ignorecase = trash;Deleted *
  fts = xapian
  fts_autoindex = yes
  fts_autoindex_exclude = \Trash
  fts_decoder = decode2text
  fts_enforced = yes
  fts_xapian = partial=3 full=20 verbose=1 lowmemory=0
  sieve = /home/user-data/mail/sieve/%d/%n.sieve
  sieve_after = /home/user-data/mail/sieve/global_after
  sieve_before = /etc/dovecot/sieve-spam.sieve
  sieve_before2 = /home/user-data/mail/sieve/global_before
  sieve_dir = /home/user-data/mail/sieve/%d/%n
  sieve_redirect_envelope_from = recipient
}
postmaster_address = postmaster@box.jammytest.com
protocols = " imap lmtp sieve pop3"
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0666
    user = postfix
  }
}
service decode2text {
  executable = script /usr/libexec/dovecot/decode2text.sh
  unix_listener decode2text {
    mode = 0666
  }
  user = dovecot
}
service imap-login {
  inet_listener imap {
    address = 127.0.0.1
    port = 143
  }
}
service indexer-worker {
  executable = /usr/bin/nice -n 10 /usr/lib/dovecot/indexer-worker
  vsz_limit = 2 G
}
service lmtp {
  inet_listener lmtp {
    address = 127.0.0.1
    port = 10026
  }
}
service pop3-login {
  inet_listener pop3 {
    port = 0
  }
}
ssl = required
ssl_cert = </home/user-data/ssl/ssl_certificate.pem
ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl_client_ca_dir = /etc/ssl/certs
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocol lmtp {
  mail_plugins = " fts fts_xapian sieve"
}
protocol imap {
  mail_max_userip_connections = 40
  mail_plugins = " fts fts_xapian antispam"
}
protocol pop3 {
  mail_plugins = " fts fts_xapian antispam"
}

If I come across the original test I had above I'll report back here.

grosjo commented 2 years ago

You need to set mail_uid mail_gid according to your config (i.e. to the owner of the folder /home/user-data/mail/mailboxes

grosjo commented 2 years ago

@jvolkenant problem solved ?

grosjo commented 2 years ago

@jvolkenant I beleive this is solved. Feel free to reopen if not