BUG: accept IMAP unsolicited responses

gekrap commented 1 year ago

Affected version:

I was trying the new UNSEEN search condition branch in a client, and added some very rough logging of the lines that are actually transmitted between server and client. patch file

Steps to reproduce:

Start with a clean mailbox, i.e. all messages are flagged "seen"
Start an imaps connection
select INBOX

Start polling the server for unseen messages via

search_condition.push_back(mailio::imaps::search_condition_t(mailio::imaps::search_condition_t::UNSEEN));
conn_imap->search(search_condition, unseen_msg_numbers);

Send a message to the account you're polling

Expected result:

After the next call of imap::search (see above), return that new message in unseen_msg_numbers. Compliance with RFC 3501 would also mean that I somehow get notified of the EXISTS response as a user or application build with mailio as a library. (Maybe related to #46?)

Observed result:

This is the complete log of server-client communication plus any mailio errors that are thrown on the way:

>>> * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS LOGINDISABLED] Ready.
<<< 1 STARTTLS
>>> 1 OK Begin TLS negotiation now.
<<< 2 LOGIN xxxx yyyyyyyyyyyyyyyyyy
>>> 2 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE] Logged in
<<< 3 SELECT "inbox"
>>> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft Junk NonJunk $MDNSent $has_cal)
>>> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft Junk NonJunk $MDNSent $has_cal \*)] Flags permitted.
>>> * 648 EXISTS
>>> * 0 RECENT
>>> * OK [UIDVALIDITY 1642511953] UIDs valid
>>> * OK [UIDNEXT 1016] Predicted next UID
>>> * OK [HIGHESTMODSEQ 2207] Highest
>>> 3 OK [READ-WRITE] Select completed (0.001 + 0.000 secs).
<<< 4 SEARCH UNSEEN
>>> * SEARCH
>>> 4 OK Search completed (0.001 + 0.000 secs).
<<< 5 SEARCH UNSEEN
>>> * SEARCH
>>> * 649 EXISTS
mailio::imap_error: Search mailbox failure.
<<< 6 SEARCH UNSEEN
>>> 5 OK Search completed (0.003 + 0.000 + 0.002 secs).
mailio::imap_error: Parsing failure.
<<< 7 SEARCH UNSEEN
>>> * SEARCH 649
>>> 6 OK Search completed (0.001 + 0.000 secs).

The 'tag mismatch' between SEARCH command and response will go on like this forever, and increment with every unsolicited response, be it another EXISTS, RECENT, or FETCH response (the latter e.g. after an old message is marked unread with another client.) I've tested a second server more or less with the same result, other than the fact that the second server sends an EXISTS and a RECENT response for every received message, so each increments the tag mismatch by 2.

karastojko commented 1 year ago

Thanks for the detailed report. Probably you are right, there must be a notification mechanism in the IMAP. That's something I have to deal with - not a trivial change. Let me finish adding the tests for the MIME format, release the new version, and I am starting with this and few other IMAP features.

gekrap commented 1 year ago

I think I found the reason for the tag mismatch:

search() throws an imap_error("Search mailbox failure.") when encountering the EXISTS response, because obviously there's no SEARCH keyword. That means that in the buffer for receive() there's still the tag OK response that doesn't get read, but is left for the next receive() call to read, which then happens to expect the responses to the next SEARCH request, but gets responses to the last one, which again leads to an error being thrown, which leads to even more uncared-for responses clogging the receiving buffer.

karastojko commented 1 year ago

Thanks for the info, I will investigate it.

karastojko commented 1 year ago

I managed to reproduce the issue and the fix is here. It is not related to the missing idle feature.

karastojko / mailio