search

ruby / net-imap

Ruby client api for Internet Message Access Protocol
https://ruby.github.io/net-imap
Other
56 stars 29 forks source link

Support for IMAP4rev2 and modern extensions #12

Open nevans opened 3 years ago

nevans commented 3 years ago

This is a meta-ticket to track support for IMAP capabilities. I'm also listing capabilities that are already supported by Net::IMAP as well as extensions that are obsolete or unsupported by IMAP servers, so that (when completed) the list will be exhaustive. Later, we can copy the list of supported RFCs and extensions into the rdoc.

This list hasn't been carefully audited yet. Some unchecked items can probably be checked off without any extra effort. As tickets and PRs are created for various features, I'll update this list to include them.

Basic protocol specifications

IANA registries

Registry references
Capabilities RFC3501
Mailbox attributes RFC3501
SASL Mechanisms RFC4422
LIST EXTENDED RFC5258
Response codes RFC5530
Keywords RFC5788, RFC8621

Commonly supported extensions

This is not an exhaustive list of all extensions, but an opinionated selection of the most important extensions that Net::IMAP should support. It is partially based on CAPABILITY data for hundreds of thousands of IMAP4 accounts, and partially based on subjective judgement. Additionally, any RFCs we already support, even if only partially, have been promoted to this list. Some extension behaviors are out-of-scope for Net::IMAP, so "support" simply means parsing and adding the necessary documentation for users of net-imap to implement the extension bevavior. Please comment below if you think something should be promoted to this list!

sorted by original RFC year

Additional standard recommendations

Other extensions

Some of these are easy to add, and many of them would be very useful. I placed them down here instead of prioritizing them above mostly due to a quick and unscientific sampling of email accounts and server capabilities.

sorted by estimated current server support

By my estimation, the following are currently supported by relatively few email addresses. They are sorted by RFC number.

New specifications

Looking at recent standards and others that are still in draft form (as of 2021-10-19):

IETF "extra" WG (Email mailstore and eXtensions To Revise or Amend)

Deprecations?

We might want to remove obsolete authentication mechanisms from the default set of authenticators. For people who still need them, the code could be preserved in separate files, with explicit requires, e.g:

require "net/imap/sasl/cram-md5_authenticator"
require "net/imap/sasl/digest-md5_authenticator"
require "net/imap/sasl/login_authenticator"
nevans commented 2 years ago

n.b. IMAP4rev2 (RFC9051) was released on 2021-08-26.

I didn't have as much time earlier this year as I'd hoped, but I'll have some more time to devote to this next week and the week after, so hopefully I can make good progress towards updating our parser for IMAP4rev2. I'd like to see what can be accomplished before RubyConf 2021! :)

nevans commented 2 years ago

n.b. I lightly modified RFC-9051's ABNF to valid ABNF syntax, and I generated railroad diagrams in unicode line-art, using kgt: https://gist.github.com/nevans/ae5a82bb81c1595b06b9e167a0376924

e.g.

response:
    │├─────╭─────────────────────────╮── response-done ─────┤│
           │                         │
           ╰──╮── continue-req ───╭──╯
              │                   │
              ╰── response-data ──╯

response-data:
    │├───── "*" ── SP ──╮── resp-cond-state ──╭── CRLF ─────┤│
                        │                     │
                        ╰─── resp-cond-bye ───╯
                        │                     │
                        ╰─── mailbox-data ────╯
                        │                     │
                        ╰─── message-data ────╯
                        │                     │
                        ╰── capability-data ──╯
                        │                     │
                        ╰──── enable-data ────╯

response-done:
    │├──╮── response-tagged ──╭──┤│
        │                     │
        ╰── response-fatal ───╯

response-fatal:
    │├───── "*" ── SP ── resp-cond-bye ── CRLF ─────┤│

response-tagged:
    │├───── tag ── SP ── resp-cond-state ── CRLF ─────┤│

and so on...

I spent a little bit of time this week going through the collection of response parser patches I've been using, to figure out which ones to convert into PRs and which ones to throw away. I hope to start on the big RFC9051 updates next week.

In most cases, the ABNF in RFC9051 seems like an easier starting point than RFC3501, because it already incorporates many of the updates from various popular extensions. And it also includes simpler extension points for the extensions which aren't included. Likewise, I think it will be simpler to retain backwards compatibility with an RFC9051 parser than to force forward compatibility into an RFC3501 parser.

nevans commented 2 years ago

Updated the checklist with links to #55 and #54. Hopefully I'll starting ticking things off this checklist soon, and not only adding things to it! 😆

Neustradamus commented 2 years ago

To follow :)

Neustradamus commented 2 years ago

Good news, it is official for TLS 1.3 Binding!

Details:

Thanks to @singpolyma who has done a PR in net-sasl for SCRAM:

nevans commented 2 years ago

Thanks that's great. I'm (slowly, sporadically) working on merging that PR and hope to finish very soon. But let's discuss SCRAM-* SASL mechanisms in #54, maybe open a new issue (or PR! 😉) for new features like channel binding, and reserve talk on this issue to be about the meta-ticket: the IMAP4rev2 checklist and extensions that are in one of the specific IMAP IANA registries, e.g: IMAP capabilities.

ssunday commented 2 years ago

I'm planning on raising a PR soon™ to incorporate the XOAUTH2 authenticator. 👍🏻