foxcpp / maddy

✉️ Composable all-in-one mail server.
https://maddy.email
GNU General Public License v3.0
5.13k stars 249 forks source link

0.6.x breaks caching clients that connected to 0.5.x in the past #512

Closed foxcpp closed 2 years ago

foxcpp commented 2 years ago

For some reason, clients do not show all messages if the database was created in 0.5. Freshly created databases in 0.6 do not have the same issue.

Different clients show different messages, maddy imap-msgs correctly shows all messages. If a new message is added, some (?) clients refresh and show all messages again.

foxcpp commented 2 years ago

Excerpt from protocol trace with Thunderbird client:

42 select "INBOX"
...
* 17 EXISTS
* 17 RECENT
...
43 UID fetch 1:* (FLAGS)
* 1 FETCH (FLAGS (\Recent \Recent) UID 29)
* 2 FETCH (FLAGS (\Recent \Recent) UID 30)
* 3 FETCH (FLAGS (\Recent \Recent) UID 31)
* 4 FETCH (FLAGS (\Recent \Seen \Recent) UID 32)
* 4 FETCH (FLAGS (\Recent \Seen \Recent) UID 37)
* 4 FETCH (FLAGS (\Recent \Seen \Recent) UID 38)
* 4 FETCH (FLAGS (\Recent \Seen \Recent) UID 39)
* 4 FETCH (FLAGS (\Recent \Seen \Recent) UID 40)
* 4 FETCH (FLAGS (\Recent \Seen \Recent) UID 41)
43 OK UID FETCH completed

Horribly broken in multiple obvious ways (duplicate sequence numbers, UID not sorted)

cbrake commented 2 years ago

looks like you're on it! That explains why I'm seeing issues you did not see in testing.

foxcpp commented 2 years ago

I pushed the commit to master that should fix the issue.

The issue was caused by go-imap-sql passing unsorted UIDs list to go-imap-mess which uses binary search and therefore misbehaves when given an unsorted input. The bug did not manifest itself in 0.6 testing because regular server operation keeps the database tables sorted already, however 0.5->0.6 schema upgrade reordered some UIDs, revealing the issue.

cbrake commented 2 years ago

working here -- thanks!