Maildir storage backend

[emersion]

Probably maildir

[xeoncross]

https://github.com/flashmob/go-guerrilla/tree/master/backends implemented multiple backends using a Backend interface just as you did with your https://github.com/emersion/go-smtp#server

[foxcpp]

What's up with https://github.com/emersion/go-imap-disk? Can we use it? (of course, we need to finish it first)

[emersion]

We can use it, but it doesn't use maildir, it uses a bolt database. This is fine too, but a maildir backend would probably be higher priority for compatibility (makes migrating easier).

[foxcpp]

https://github.com/luksen/maildir Except that we also need some way to store UIDs and some other meta-data (custom flags, internal date).

[lupine]

Dovecot stores UIDs in a uidlist file. I've been working on a project similar to maddy, on and off (maddy is better!), and implemented a parser for it here: https://gitlab.com/lupine/crockery/tree/master/internal/imap/uidlist

[foxcpp]

We currently want to implement support for Maildir storage by extending https://github.com/emersion/go-imap-disk which uses BoltDB for all meta-data. Personally, I don't like the idea of supporting other servers internal formats in maddy. Also we can't use your code anyway due to incompatible licenses (crockery's AGPL and maddy's MIT).

[emersion]

Are the UIDs the only metadata we need to save?

[foxcpp]

Mailbox-related: UIDVALIDITY, UIDNEXT Message-related: flags, RFC822 size [1], internal date [2], UID

[1] - Probably we would have to store message using native line separators to reduce possible interoperability problems [research needed], but we should still report RFC822 size with CRLF (Is that right?). [2] - Probably FS "modification time"?

[emersion]

Probably we would have to store message using native line separators to reduce possible interoperability problems

No, MIME says you MUST use CRLF. Using non-CRLF line endings with MIME messages leads to various issues including DKIM signature breakage.

[foxcpp]

Related discussion: https://dovecot.org/list/dovecot/2013-May/090499.html

The original (djb) definition of Maildir assumed that messages would be written to the maildir with LF line endings, and both MTA and POP server would translate back to CRLF as needed. Dovecot (as a POP server) can deal with messages in either format, and should always return them to clients with CRLF.

It mentions POP, but I think it applies to IMAP too.

[emersion]

Gah that is annoying.

Also note that:

mail_save_crlf setting controls if mails are saved as CRLF or LF to storage.

[foxcpp]

Started some work here. https://github.com/foxcpp/go-imap-maildir

Not high priority though, go-imap-sql is working reasonably well.

[foxcpp]

go-imap-maildy is ready for use. Integration is blocked by #259 since it was written for v2.

[foxcpp]

Blocked by https://github.com/foxcpp/go-imap-maildir/issues/1 since we need to use go-imap-mess "external update" adapter for maddyctl synchronization.

[iacore]

There is this better idea that improves upon maildir called m2dir: https://bitfehler.srht.site/posts/2024-04-15_m2dir-treating-mails-as-files-without-going-crazy.html

[foxcpp]

go-imap-maildir probably could be refactored to actually support both formats.

[foxcpp]

However, currently the priority is the development of the new version of imapsql (internally named maddy-storage). So unless someone steps up, maildir or m2dir won't be supported for the foreseeable future.