search

ceph-dovecot / dovecot-ceph-plugin

Dovecot plugin for storing mails in a Ceph cluster
Other
131 stars 23 forks source link

delete mailbox / unreferenced objects #255

Open jrse opened 5 years ago

jrse commented 5 years ago

During imaptests, in our "test production" cluster we observed several mail objects which where no longer referenced in the mailbox index file. A reason for this may be, that we delete the mail reference from the users mailbox index before we made sure that the object in the mailstore is deleted. We do this to minimize the time objects, which are marked as delete, to remain in the index file.

In concurency situation where we have several imap processes per user which are reading/deleting the same mail, this may result in reading a already deleted file. SDBOX and rbox handle this situation by marking the deleted file as deleted.

To prove that the unreferenced objects are mails, which where not deleted due to testcluster/or imaptest failures, we need to gather more information. To check for unreferenced objects, there exist the doveadm rmb check indices command, which allows to find unreferenced objects.

If this proves true, one solution is to wait until mails which where marked as expunged are really deleted, before removing them from the index.

jrse commented 5 years ago

storage module imapc could also give some hints, how to handle this situation.