Inbox folder shown twice

[mattfbacon]

I saw #66 but I'm not sure if this is the same thing, because it doesn't seem to be specific to any one mail client. I tried with Roundcube (webmail) and Fair Email (android app) and they both show Inbox twice. It's strange to me because when I manually execute a LIST command over IMAP, I see all the folders once as they should be:

import imaplib
mb = imaplib.IMAP4_SSL('<new server>', 993)
mb.login(...)
for x in mb.list()[1]:
  print(x)

b'(\\Archive) "/" Archive'
b'() "/" Cherished'
b'(\\Drafts) "/" Drafts'
b'() "/" Inbox'
b'(\\Junk) "/" Junk'
b'() "/" Rejects'
b'(\\Sent) "/" Sent'
b'(\\Trash) "/" Trash'

I checked on my old mail server (dovecot) and got this:

b'(\\HasNoChildren \\UnMarked) "/" Cherished'
b'(\\HasNoChildren \\UnMarked) "/" Archive'
b'(\\HasNoChildren \\Trash) "/" Trash'
b'(\\HasNoChildren \\Junk) "/" Junk'
b'(\\HasNoChildren \\UnMarked \\Sent) "/" Sent'
b'(\\HasNoChildren \\UnMarked \\Drafts) "/" Drafts'
b'(\\HasNoChildren) "/" INBOX'

The only difference I see (other than the HasNoChildren and UnMarked flags) is that the Inbox folder is listed as INBOX in all caps. I'm not sure if this would make any difference in the email clients.

[mjl-]

Thanks for reporting. I think you've got it diagnosed. I installed roundcube locally for testing (wanted to do that at some point, this nudge helped). If I change the name to "INBOX" in the IMAP response, roundcube no longer shows the duplicate.

I'm pretty sure the INBOX mailbox is very special in IMAP, and has to be treated case-insensitively. I suspect roundcube and Fail Email are not comparing case-insensitively. Ideally, this would be fixed in those clients. But I suppose it is more foolproof to just have INBOX in the IMAP response in mox. I'll see if I can add it somewhat cleanly.

For reference, INBOX being case-insensitive has been the case forever. See the mailbox ABNF syntax at the very old https://datatracker.ietf.org/doc/html/rfc1730#page-58

   mailbox         ::= "INBOX" / astring
                       ;; INBOX is case-insensitive; other names may be
                       ;; case-sensitive depending on implementation.

And the more recent https://datatracker.ietf.org/doc/html/rfc9051#name-formal-syntax

mailbox         = "INBOX" / astring
                    ; INBOX is case insensitive.  All case variants
                    ; of INBOX (e.g., "iNbOx") MUST be interpreted as
                    ; INBOX, not as an astring.  An astring that
                    ; consists of the case-insensitive sequence
                    ; "I" "N" "B" "O" "X" is considered
                    ; to be an INBOX and not an astring.
                    ; Refer to Section 5.1 for further
                    ; semantic details of mailbox names.

In any case it would be good to report this to roundcube and fair email.

[mjl-]

I didn't know about Fair Email, but installed it on a test-android device just now. But I'm not seeing a duplicate Inbox folder. I'm not familiar with the application. Where are you seeing this behaviour?

[mattfbacon]

In the hamburger menu you can click on an individual account to see its folders. I did this for one of my accounts and you can see here how Inbox is repeated:

[mattfbacon]

And I will open issues on Roundcube and Fair Email. But I agree that if possible we should change this in mox too, for better compatibility.

[mjl-]

Ah, that is where I'm looking to, but I'm not seeing that behaviour.

I'm testing with Fair Email 1.2103a, and essentially mox v0.0.7. Perhaps there is something else different with our setups. I'll try to get the patch in, perhaps you can then test with Fair Email if it makes the problem go away.

[mattfbacon]

I'm chatting with the FairEmail maintainer and we figured it out. It had to do with my transition from the old IMAP server (dovecot) to the new one (mox). Since it saw "INBOX" before and "Inbox" now, it thought they were two different folders so it showed it twice, but both with the same content (because the mox IMAP server correctly treats them as the same folder).

So for someone else doing the same thing as me, the patch to make mox say "INBOX" would have fixed the issue, but ultimately I still think it's a bug with FairEmail because it should have compared the old and new Inbox folders case-insensitively and realized that they're actually the same folder.

[mjl-]

That's quick, thanks! I worked on a patch to always output INBOX to clients. But it would cause corner-case buggy behaviour exhibited by mox. Admittedly very corner-case, but mox does already have a test for it. By sending back INBOX, that is the implied canonical name. While the actual canonical name is Inbox. Now when a client tries to match "INBO*", it would at least expect to match "INBOX". But with the approach of translating Inbox to INBOX on IMAP responses only, that path wouldn't match against the canonical Inbox. I don't think there is a requirement for pattern matching to be checked case-insensitively against mailbox names. See https://github.com/mjl-/mox/blob/7dce883097972eccdf19c027bad7766975a5644c/imapserver/list_test.go#L53.

As it also seems this is something that should be fixed by clients, let's try to leave it with them for now.

[mattfbacon]

The FairEmail maintainer made a commit that he believes will fix the issue: https://github.com/M66B/FairEmail/commit/08855c7b64c755351e781e3ef9ea97016314c73e?w=1

[mjl-]

Seems roundcube also has a patch merged, https://github.com/roundcube/roundcubemail/issues/9166. Thanks for reporting!