search

marlam / msmtp

SMTP client with sendmail compatible interface
https://marlam.de/msmtp
GNU General Public License v3.0
173 stars 37 forks source link

SCRAM-SHA-1(-PLUS) + SCRAM-SHA-256(-PLUS) + SCRAM-SHA-512(-PLUS) + SCRAM-SHA3-512(-PLUS) supports #36

Closed Neustradamus closed 5 months ago

Neustradamus commented 3 years ago

CRAM-MD5 to Historic:

RFC6331: Moving DIGEST-MD5 to Historic:

RFC 8600: "When using the SASL SCRAM mechanism, the SCRAM-SHA-256-PLUS variant SHOULD be preferred over the SCRAM-SHA-256 variant, and SHA-256 variants [RFC7677] SHOULD be preferred over SHA-1 variants [RFC5802]".

SCRAM-SHA-1(-PLUS):

SCRAM-SHA-256(-PLUS):

SCRAM-SHA-512(-PLUS):

SCRAM-SHA3-512(-PLUS):

-PLUS variants:

LDAP:

HTTP:

2FA:

IANA:

Note, after SCRAM-SHA-1(-PLUS):

Linked to:

marlam commented 3 years ago

I have no plans to implement any of this, for the following reason:

Nowadays every SMTP session must be protected with TLS. When TLS is active, there is no reason to use any SCRAM* method. (RFC 5802 says "there are some significant security concerns" with transmitting passwords over TLS-secured connections, but does not list any of them, and I have not seen a compelling argument for that statement yet.)

I therefore consider all methods except PLAIN, OAUTHBEARER, EXTERNAL (for TLS client certificates), LOGIN (strictly only for compatibility with Microsoft crap), and maybe GSSAPI (only in special situations) to be useless today.

Some of the deprecated methods like CRAM-MD5 etc are only left for backward compatibility and will probably be removed in the next larger update (1.10 or 2.0), so their presence is no reason to add SCRAM* methods.

So unless someone can convince me that any SCRAM method over TLS offers a real* (and not just theoretical) advantage over the simpler methods listed above, I will not add support for them.

marlam commented 3 years ago

Simon Josefsson convinced me that msmtp should support SCRAM-SHA-256. It now does.

However, I am still convinced that the SCRAM-* methods are just an unnecessary complexity and offer no real benefit.

Neustradamus commented 3 years ago

@marlam: Thanks for your answer :)

@jas4711: Thanks a lot for your SCRAM-SHA-256 improvements in msmtp!

Can you look for mpop too, to have same into mpop and msmtp?

@marlam has not created a mimap yet, maybe soon?

@marlam: SCRAM-SHA-* is the evolution of DIGEST-MD5, a real security, please read the RFC6331.

But the PR is not merged and it is closed:

And I do not find SCRAM-SHA-256:

marlam commented 3 years ago

Hi!

Can you look for mpop too, to have same into mpop and msmtp? [...] But the PR is not merged and it is closed:

Hi, I applied Simon's patch to both msmtp and mpop already. The reason you did not see it yet on Github is that the main repository is synced to the Github mirror only once per day. It should be there by now.

@MarlaN has not created a mimap yet, maybe soon?

What's a mimap?

@marlam: SCRAM-SHA-* is the evolution of DIGEST-MD5, a real security, please read the RFC6331.

I fully understand hat DIGEST-MD5 is terrible and obsolete; that's not my point. My point is that I don't see why anyone would need SCRAM*. Everybody uses TLS for everything today (or should be), and if that is the case, I see no practical benefit of SCRAM* vs good ol' PLAIN authentication:

I may be wrong about these points, and if so, I would welcome to be corrected here. If you disagree, please give more information or a link.

Neustradamus commented 3 years ago

@marlam: Thanks for your reply! Sorry for "N", I have edited.

I have not seen the @jas4711 PR here for mpop: https://github.com/marlam/mpop-mirror/pulls?q=is%3Apr It was directly in the main original source ^^

mimap would be an IMAP client, like mpop is a POP3 client ;) It will be really nice...

With SCRAM, the password on the server is not in PLAIN, better security, confidentiality.

marlam commented 3 years ago

mimap would be an IMAP client, like mpop is a POP3 client ;) It will be really nice...

Ah, ok. I'd like to do that, see here for a first thought: https://github.com/marlam/mpop-mirror/issues/4

With SCRAM, the password on the server is not in PLAIN, better security, confidentiality.

Sorry, not sufficient:

To put it in other words, I'd like an answer to the question "In what way is SCRAM-SHA-256 better than PLAIN for password-based authentication in the context of a TLS-secured single-purpose protocol like IMAP or SMTP?" My current answer would be: in no way that is of practical relevance.

Neustradamus commented 3 years ago

No with SCRAM the password is not saved into a database in clear.

Time to look for IMAP, you can see the last Internet-Draft:

I have added mpop and msmtp in the list, but you have not done releases yet.

Note:

marlam commented 3 years ago

No with SCRAM the password is not saved into a database in clear.

And it is not with PLAIN either!

* GNU SASL (gsasl) supports SCRAM-SHA-256(-PLUS) since 1.9.1: http://git.savannah.gnu.org/gitweb/?p=gsasl.git;a=blob;f=NEWS;hb=HEAD

* Dovecot supports SCRAM-SHA-256(-PLUS) since 2.3.10: https://doc.dovecot.org/configuration_manual/authentication/authentication_mechanisms/

* Cyrus SASL/IMAP supports SCRAM-SHA-256(-PLUS) and more since 2.1.27: https://www.cyrusimap.org/sasl/sasl/authentication_mechanisms.html

I know that these support SCRAM-SHA-256, but that does not answer the question "why would anyone want to use it instead of the much simpler PLAIN-over-TLS".

This is going nowhere. I must conclude that there is indeed no benefit to SCRAM.

marlam commented 3 years ago

After receiving feedback on an article I wrote about my doubts about SCRAM, I changed my position. SCRAM is indeed useful. See this update for the reasons that changed my mind.

jas4711 commented 3 years ago

Thanks for update! Big of you to reconsider. To be honest, I'm somewhere between your extreme positions. I do believe PLAIN is not ideal and that we can do better, and SCRAM is not bad. The complexity introduced with SCRAM over PLAIN is only worth it if it actually solves the problems we want to address: however, SCRAM does not REQUIRE servers to store passwords salted/hashed, and quite often they are stored in cleartext, which is WORSE than with PLAIN where deployment actually have started to have salted/hashed passwords fairly often. Secondly, the tls-unique channel binding is broken and does not offer the security guarantees that people thought. The idea of channel bindings is still good, IMHO, but there is two problems in SCRAM with it: 1) SCRAM chose to demand a broken channel binding, and updating it is not easy today, and 2) it was optional, so in practice its use won't be enforced. Finally, the unfortunate choice of having SCRAM be a family of mechanisms leads to plurality of hash methods being suggested which is detrimental to deployment.

Neustradamus commented 3 years ago

Thanks for your changes about SCRAM-SHA-*.

I think the order is not good here:

And CRAM-MD5 and DIGEST-MD5 are less good than PLAIN.

20 November 2008: CRAM-MD5 to Historic:

29 June 2017: CRAM-MD5 to Historic:

July 2011: RFC6331: Moving DIGEST-MD5 to Historic:

jas4711 commented 3 years ago

Closing the circle here, I blogged about SCRAM concerns here: https://blog.josefsson.org/2021/06/08/whats-wrong-with-scram/

marlam commented 3 years ago

Thank you for this article Simon!

Neustradamus commented 2 years ago

@marlam: We can thank @jas4711 who has worked on the support of the new security RFC in GNU SASL:

Little details, to know easily:

If you can test before merging and add support in msmtp?

Thanks in advance.

Neustradamus commented 11 months ago

@jas4711, @marlam: I think that you have seen the jabber.ru MITM:

marlam commented 11 months ago

I am currently adding SCRAM-*-PLUS to msmtp (and later to mpop), using gnutls_session_channel_binding() to get the channel binding information, type EXPORTER for TLS 1.3 and UNIQUE for TLS <= 1.2.

I have a question regarding the AUTH capabilities advertised in a server greeting. Given that the SCRAM- methods are able to protect against certain MITM attacks, Is it not a problem that a MITM could remove all the SCRAM- methods from the server greeting, thereby making the client fall back to PLAIN or similar?

In other words, does a user have to enforce using a SCRAM-* method to benefit from its protections?

jas4711 commented 11 months ago

It was some time since I was well versed on the SCRAM downgrade protection complexity, but I believe that generally you are right that an adversary is able to MITM the TLS connection and make data modifications, AND if PLAIN is acceptable to the user/application, there is nothing SCRAM can do to protect against this.

What SCRAM-PLUS can offer is the ability to DETECT (and abort) passive TLS MITM's, since the channel binding will fail. If both client and server supports SCRAM+SCRAM-PLUS, then even SCRAM will allow detection (and abort) of active TLS attacks.

So ideally you would want to enforce at least SCRAM to any server that at some point offers SCRAM, and ideally anything less than SCRAM should not ever be acceptable to users/applications by default. I do understand that the e-mail world is far behind the HTTPS world here though, so this is usually not that feasible.

jas4711 commented 11 months ago

To compare with the jabber.ru issue: if clients (and servers) had supported SCRAM-PLUS, then you would have gotten connection failures and notice the passive TLS MITM before the certificate expired. The attacker could turn active and remove SCRAM+SCRAM-PLUS but this would be more costly than a pure passive attack.

marlam commented 11 months ago

Thank you for this information, it helps.

So in a first step, to not break existing setups, users of msmtp and mpop will have to manually select a SCRAM-* method to prevent these kinds of downgrade attacks.

In a second step, I'd like to add a simpler flag that enforces SCRAM-*[-PLUS] authentication without requiring the user to select one specific method manually, and recommend that flag prominently in all documentation and examples.

In the future, when/if SCRAM- is widely supported in servers, this flag should then become the default, but that would break setups where the server does not support SCRAM-...

Neustradamus commented 11 months ago

@marlam: Thanks for your comments and thanks to @jas4711 for comments too :)

I take the opportunity to say that I have requested to @jas4711 to add "tls-server-end-point" support in GSASL too:

In the specified ticket, I have added links to XEPs about SCRAM.

jas4711 commented 11 months ago

I would ignore tls-server-end-point and focus on mandatory SCRAM-PLUS with tls-exporter. If there is ever a serious significant use-case for non-plus SCRAM or tls-server-end-point, it could be added later, but I believe adding complexity to support them is counter-productive to reach a SCRAM-PLUS with tls-exporter world.

Neustradamus commented 11 months ago

@jas4711, it is in XEPs:

Neustradamus commented 11 months ago

@marlam: @mbhangui has added supports in indimail-mta, he has done talks with @jas4711 too:

Manvendra has helped @schengawegga for Net_SMTP too.

mbhangui commented 11 months ago

If there is any I can help, please let me know. I can provide a setup that provides accounts supporting SCRAM and SCRAM PLUS methods for authentication.

Regards Manny

On Thu, 16 Nov, 2023, 17:11 Neustradamus, @.***> wrote:

@marlam https://github.com/marlam: @mbhangui https://github.com/mbhangui has added support in indimail-mta, he has done talks with @jas4711 https://github.com/jas4711 too:

Manvendra has helped @schengawegga https://github.com/schengawegga for Net_SMTP too.

— Reply to this email directly, view it on GitHub https://github.com/marlam/msmtp-mirror/issues/36#issuecomment-1814281030, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6RQLP3F7PWBKHWHMBGJMDYEX3VZAVCNFSM4TJNJOCKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRGQZDQMJQGMYA . You are receiving this because you were mentioned.Message ID: @.***>

marlam commented 10 months ago

Support for SCRAM-SHA-256-PLUS and SCRAM-SHA-1-PLUS is now in the git repository.

This is tested so far against Exim via TLS 1.2 and TLS 1.3, but this basically tests a GnuTLS/GSASL client against a GnuTLS/GSASL server, so any additional testing is very welcome.

A future patch will likely introduce auth=scram-plus and auth=scram support so that the user at least does not have to care about the hash-du-jour to get the security benefits of the scram methods.

marlam commented 10 months ago

The same code is now also in mpop for POP3. If you can test this, please do.

jas4711 commented 10 months ago

Support for SCRAM-SHA-256-PLUS and SCRAM-SHA-1-PLUS is now in the git repository.

This is tested so far against Exim via TLS 1.2 and TLS 1.3, but this basically tests a GnuTLS/GSASL client against a GnuTLS/GSASL server, so any additional testing is very welcome.

Yay! Dovecot support SCRAM (but not SCRAM-PLUS, I think) and is fairly easy to interop-test against: https://gitlab.com/gsasl/gsasl/-/blob/b21aa50acebcf1350832aba4af8fc7f84bb68f3b/.gitlab-ci.yml#L651

A future patch will likely introduce auth=scram-plus and auth=scram support so that the user at least does not have to care about the hash-du-jour to get the security benefits of the scram methods.

Makes sense, although I think it is one of the disadvantages with SCRAM: it leads to UX complexity like this, instead of just providing secure defaults and reduce UX complexity. Web browser security has solved that, but it seems SMTP/IMAP/POP3/XMPP is stuck in older design patterns. No fault of msmtp/mpop, of course, but just a general observation.

/Simon

Neustradamus commented 10 months ago

@marlam: Thanks for your improvements!

@jas4711: @stephanbosch has added SCRAM in Dovecot and he has commented here:

At the same time, you can see tls-server-end-point (in more tls-unique and tls-exporter) in xmpp lib used ejabberd...

marlam commented 10 months ago

I have more questions about SCRAM...

I was told that if a server stores the salted and hashed password, then stealing that allows impersonating the client, so it's no better than storing clear text passwords.

Then what is the server supposed to store instead?

In GSASL, what is SERVERKEY and STOREDKEY, and how do they relate to SCRAM_SALTED_PASSWORD? All of those seem to be derived from PASSWORD using SCRAM_ITER and SCRAM_SALT.

jas4711 commented 10 months ago

I hope that the manual covers this, see: https://www.gnu.org/software/gsasl/manual/html_node/SCRAM.html#SCRAM

Yes it is preferential to store SERVERKEY/STOREDKEY on the server, however I believe this advantage is exagerated: if you steal them, you only need one active MITM-attack to be able to compute the password-equivalents anyway. I think this is another example of were the SCRAM design introduces complexity that have questionable security advantage in the real world. The complexity required to support SERVERKEY/STOREDKEY, and security concerns as a consequence, may be larger than the slight theoretical gain that is won.

But does msmtp/mpop implement a server?

marlam commented 10 months ago

Msmtp/mpop both come with tiny servers, but these are intended only as a local bridge between a mail client such as Thunderbird and msmtp/mpop, so SCRAM is not relevant for them.

I asked because as far as I can see, both Exim and Dovecot can only work with either plaintext or the hashed/salted password, not with SERVERKEY/STOREDKEY. So it seems offering SCRAM on a mail server today involves some tradeoffs...

jas4711 commented 10 months ago

Msmtp/mpop both come with tiny servers, but these are intended only as a local bridge between a mail client such as Thunderbird and msmtp/mpop, so SCRAM is not relevant for them.

I asked because as far as I can see, both Exim and Dovecot can only work with either plaintext or the hashed/salted password, not with SERVERKEY/STOREDKEY. So it seems offering SCRAM on a mail server today involves some tradeoffs...

I see. I think using hashed/salted passwords on the server is acceptable today -- if you are worried about that, the theoretical gain offered by moving to SERVERKEY/STOREDKEY is small, and you are better of moving to Kerberos, SRP or the forthcoming OPAQUE mechanism. Doing one failed authentication attempted using stolen SERVEKEY/STOREDKEY as an attacker is a cheap way to get a password-equivalent.

I think SERVERKEY/STOREDKEY is mostly a complexity distraction that causes people to not implement SCRAM because they think they need to implement SERVERKEY/STOREDKEY too. So they stick with CRAM-MD5 or TLS+PLAIN.

/Simon

gowthamgts commented 6 months ago

careful here - https://news.ycombinator.com/item?id=39868682. the author has been pushing for lot of sec related changes in lot of repositories.

Neustradamus commented 6 months ago

@gowthamgts: Do you attack me because I have requested the XZ update? I am not linked to the XZ project.

Neustradamus commented 6 months ago

@gowthamgts: Strange, your messages are not clear:

You can follow my announcements here:

The good point, people speak about SCRAM "Salted Challenge Response Authentication Mechanism" security ;)

Badly, some people or projects like only old unsecure mechanisms, some would like security improvements.

LeoniePhiline commented 6 months ago

@Neustradamus Don't @ me.

Neustradamus commented 6 months ago

@LeoniePhiline: Sorry, thanks for your message. It is important to understand all and not a little part :) I have edited my messages.

marlam commented 5 months ago

SCRAM support is now complete from my point of view, at least for now. Let's see if somebody actually uses it and provides useful feedback :)

Neustradamus commented 5 months ago

@marlam: In part yes, not full yet. It is good for SCRAM-SHA-1(-PLUS) + SCRAM-SHA-256(-PLUS) but not for SCRAM-SHA-512(-PLUS) + SCRAM-SHA3-512(-PLUS), I can create another ticket for it.

I will see with @jas4711 for "tls-server-end-point" in gsasl, the missing part.

jas4711 commented 5 months ago

I think SCRAM-SHA512, SCRAM-SHA3 and tls-server-end-point are all harmful to the overall ecosystem security of SCRAM. They don't offer anything convincing compared to what we already have, and adding more SCRAM-parametrization adds complexity and maintenance costs, which lead to worse security.