search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30k stars 3.79k forks source link

security: support SCRAM-SHA-256 authentication method #42519

Closed rolandcrosby closed 2 years ago

rolandcrosby commented 4 years ago

Summary

Add the SCRAM-SHA-256 auth method:

Protocol summary

Example SCRAM client-server exchange as per RFC 7677:

   This is a simple example of a SCRAM-SHA-256 authentication exchange
   when the client doesn't support channel bindings.  The username
   'user' and password 'pencil' are being used.

   C: n,,n=user,r=rOprNGfwEbeRWgbNEkqO

   S: r=rOprNGfwEbeRWgbNEkqO%hvYDpWUa2RaTCAfuxFIlj)hNlF$k0,
      s=W22ZaJ0SNY7soEsUEjb6gQ==,i=4096

   C: c=biws,r=rOprNGfwEbeRWgbNEkqO%hvYDpWUa2RaTCAfuxFIlj)hNlF$k0,
      p=dHzbZapWIk4jUhN+Ute9ytag9zjfMHgsqmmiz7AndVQ=

   S: v=6rriTRBi23WpRR/wtup+mMhUZUn/dB5nLTJRsjl95G4=

Summary of the protocol as per RFC 5802:

     SaltedPassword  := Hi(Normalize(password), salt, i)
     ClientKey       := HMAC(SaltedPassword, "Client Key")
     StoredKey       := H(ClientKey)
     AuthMessage     := client-first-message-bare + "," +
                        server-first-message + "," +
                        client-final-message-without-proof
     ClientSignature := HMAC(StoredKey, AuthMessage)
     ClientProof     := ClientKey XOR ClientSignature
     ServerKey       := HMAC(SaltedPassword, "Server Key")
     ServerSignature := HMAC(ServerKey, AuthMessage)

   The server authenticates the client by computing the ClientSignature,
   exclusive-ORing that with the ClientProof to recover the ClientKey
   and verifying the correctness of the ClientKey by applying the hash
   function and comparing the result to the StoredKey.  If the ClientKey
   is correct, this proves that the client has access to the user's
   password.

   Similarly, the client authenticates the server by computing the
   ServerSignature and comparing it to the value sent by the server.  If
   the two are equal, it proves that the server had access to the user's
   ServerKey.

What gets stored in the database:

   If the password is
   encrypted with SCRAM-SHA-256, it consists of 5 fields separated by colons.
   The first field is the constant <literal>scram-sha-256</literal>, to
   identify the password as a SCRAM-SHA-256 verifier. The second field is a
   salt, Base64-encoded, and the third field is the number of iterations used
   to generate the password.  The fourth field and fifth field are the stored
   key and server key, respectively, in hexadecimal format. A password that
   does not follow either of those formats is assumed to be unencrypted.

Special note about SCRAM for SQL

SCRAM specifies "n=user" is the username in the initial client message. However, in PostgreSQL the username is sent in the startup packet, and the username in the SCRAM exchange is ignored. libpq always sends it as an empty string.

postgres source code, function read_client_first_message in auth-scram.c

Special note about leaking information

If a user doesn't exist, perform a "mock" authentication, by constructing an authentic-looking challenge on the fly. The challenge is derived from a new system-wide random value, "mock authentication nonce", which is created at initdb, and stored in the control file. We go through these motions, in order to not give away the information on whether the user exists, to unauthenticated users.

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=818fd4a67d610991757b610755e3065fb99d80a5#patch11

References

Motivation

Postgres added support for SCRAM-SHA-256 password encryption in version 10. This is a more modern password authentication mechanism than the old "MD5" password authentication mechanism that pg was previously using.

Note that CockroachDB doesn't even support MD5 authentication, and instead requires the client to present the password in cleartext. This is because CRDB uses a non-standard encoding of the password hash in-DB using bcrypt.

The benefit of SCRAM is twofold:

  1. it increases overall server security, by ensuring that the server never sees cleartext passwords even during authn verification.

  2. it pushes the CPU cost of password checks to the client-side.

Note that implementing pg's native MD5 authentication would achieve the same goals, but MD5 authn is vulnerable to various attack vectors where SCRAM is not.

Epic CRDB-5349

ajwerner commented 4 years ago

SCRAM-SHA-256 pushes the expensive hashing step to happen at the time that a password is set, meaning authentication events can be faster and cheaper.

It also pushes hashing to the client on every authentication attempt, though a clients can cache the outcome of this work. The client hashing makes the protocol more resistant to offline attacks; for each password the attacker wants to try, they'll need to hash again.

knz commented 4 years ago

cc @bdarnell @aaron-crl -- I have extended/populated the top issue description with the technical details about the work to be done.

knz commented 4 years ago

RFC here: #51599

aaron-crl commented 4 years ago

cc @thtruo for tracking.

joshimhoff commented 3 years ago

I've been paged two times in the last month or so due to excess CPU usage from password hashing causing node liveness problems & thus large scale issues with a CC cluster.

  1. Are we shooting to get this done by a certain date?
  2. Once we have it, will all password authentication happen via this method?
thtruo commented 3 years ago

Thanks for the callout Josh. This is something the Security team is planning to take on, as part of the effort to lift the current authentication code into an independent package, though we don't have a specific date in mind yet.

IIRC once we have it, all pw authn should happen with SCRAM. cc @aaron-crl to shed more light

cc @piyush-singh for awareness

joshimhoff commented 3 years ago

Understood! Can you give me a sense of how big of an engineering lift it is?

Is it possible to backport this change? I kinda doubt it but why not ask.

If a user creates a username with password before this change is rolled out, are they stuck without SCRAM for ever? Or does a migration get run?

bdarnell commented 3 years ago

There are some non-trivial changes to the authentication protocol that go together with using SCRAM, so it's a substantial amount of work and wouldn't normally qualify for a backport. (one reason I'd like to see us have CC run its own forks instead of being held back by mainline releases and their backport policies)

Once we have it, will all password authentication happen via this method?

It's possible that some client drivers don't support SCRAM (or versions that support it aren't widely rolled out yet - it was a relatively recent addition and these things can move slowly). In that case we'd have to decide whether we want to cut off those clients or continue to allow non-SCRAM logins (for CC without network-level access controls, I think we'd want to go SCRAM-only, but on-prem customers might choose differently).

If a user creates a username with password before this change is rolled out, are they stuck without SCRAM for ever? Or does a migration get run?

In any case, we'd need to continue supporting non-SCRAM logins at least for a transition period. We can't do a SCRAM login when the database contains a bcrypted password, and we can't convert directly from bcrypt to SCRAM. The best migration we could do would be to convert passwords to the SCRAM format the next time they log in. That would solve the "accidental DoS" problem of a legitimate user logging in repeatedly, but would still allow password-guessing attacks to consume a lot of CPU.

knz commented 2 years ago

NB: an end-user reports (in #65117) that the Crystal pg driver does not enable cleartext auth by default, and thus without SCRAM they cannot use CockroachCloud out of the box.

Neustradamus commented 2 years ago

Linked to: