Does Jetty Support Modular Crypt Format (MCF)

[Devmond]

Jetty Version 10

Java Version 17

Question The jetty documentation around password hashing is opaque at best. Does jetty support the MCF format? I see examples of specifying CRYPT:XYX but I don't see any indication that passing hash parameters via MFC is supported.

I have also tried hashing with these functions:

org.eclipse.jetty.util.security.UnixCrypt.crypt(password,salt);
org.eclipse.jetty.util.security.Credential.Crypt.crypt(salt,password);

UnixCrypt.crypt will not accept the salt specified with the MFC prefix so that's out. looking at the Credential.Crypt.crypt source, it is doing this:

public static String crypt(String user, String pw)
{
     return "CRYPT:" + UnixCrypt.crypt(pw, user);
}

so same thing, ie UnixCrypt.crypt doesn't support MFC.

[Devmond]

correction the salt error was not from org.eclipse.jetty.util.security.UnixCrypt.crypt, however the salt when specified seems to have no effect.

[joakime]

No, Jetty's UnixCrypt does not support Modular Crypt Format (MCF).

The original MCF was a product of the passlib project (not a formal spec), and is documented by the passlib project. It saw some adoption outside of passlib around the 2014 time period.

But as others started introducing MCF into their own projects it quickly became apparent the deficiencies of MCF (and its documentation). By early 2015 those implementations were all over the place, no implementation being compatible with another implementation.

These early adopters were unhappy with MCF, and proposed all kinds of other solutions. By late 2015 MCF was already being deprecated in projects outside of unix crypt(3) C library. By 2016 it was already being deprecated and removed from other projects and replaced with other things (like PHC).

Even the Passlib project itself deprecated MCF in 2016.

[Devmond]

Thanks for the followup @joakime.

Is there any other option then for specifying a hash parameter? I am currently using MD5 but that is out of favor and I would need an option for SHA-256 or better. I looked at the jetty code and it has MD5 implementation hard coded so I suspect that's what I'll be stuck with.

If my read is correct the default CRYPT implementation uses DES, which is also considered outdated and insecure.

Is the jetty recommendation basically roll-your-own LoginService ?

[joakime]

The UnixCrypt class is used by Password and Credential classes.

It aids in on-disk storage of configurations for passwords.

Using UnixCrypt or OBF or even a SHA-256 (with or without seed) doesn't change how secure this is. If someone has filesystem level access to your configuration, they can decrypt the passwords.

Note: MD5 based configurations are for verifying a provided password (eg: Credentials) from a user. Hashes (like MD5 or SHA-256) cannot be used for things that need reversible passwords, like ssl keystore, jdbc connections, etc.

When it comes to verifying a password from a user that means we are limited to what is available in HTTP. So that means plaintext password or DIGEST auth (which is only MD5 anyway).

Two questions.

1. Where / what technology would you use this SHA-256 hash? (There's no spec in HTTP that uses SHA-256 hash for password auth)
2. And how would you specify the seed in a way where the above mentioned filesystem access issue is resolved?

[joakime]

Is the jetty recommendation basically roll-your-own LoginService ?

The LoginService is limited to the technology in the various HTTP specs. That means HTTP Authentication - see https://datatracker.ietf.org/doc/html/rfc7235

With normal HTTP, you are limited to Plaintext or Digest. Digest is documented at https://datatracker.ietf.org/doc/html/rfc2617

Which means you are limited to MD5 only. You couldn't use SHA-256 for Digest, even if you wanted to (no client supports it, not even chrome or firefox).

The improved security in HTTP Auth comes from the TLS layer, not the authentication layer.

If you don't want to use Digest auth because of MD5, then your options quickly reduce.

A common technique is specialized bearer token auth. (eg: the Authorization: Bearer <YOUR-TOKEN> request header) Which wouldn't use the LoginService, as no username or password is provided in token auth. You also wouldn't have browser support for token auth. Jetty does not have an Implementation for bearer tokens.

There is also LDAP, and SPNEGO options (Jetty has impls for those).

You can also look at JAAS and JASPI options. (Jetty has bridges for those APIs)

There is also an OpenID (OAuth) implementation available in Jetty.

The next thing you can pursue is TLS Client Certificate verification. This happens deep in the TLS layer in the JVM, and Jetty isn't involved. This is complicated to setup on a browser, and is quite specialized in technique.

[Devmond]

@joakime , my use case is for securing web app running in jetty. Basically a user creates a password, the hash (MD5:PASSWORD_HASH) is what's stored in the database and later used for authentication against user supplied values.

The concern with MD5/SHA-1 stems from the existence of rainbow tables (password:hash maps) and the efficiency with which an attacker could loop through them and test the whole table against your user table to find a matching hash and gain access.

I am not a security pro, but if the internet is to be believed, one should no longer be using weak hash algorithms for passwords.

Now a question for any security pros reading this, in regards to dealing with weak hashes and rainbow tables.

Has anyone considered pairing hashes with limited password data to counteract weak hashes?

What if the option is added to include a limited set of randomly extracted characters from the source password along with the position of said characters, then store that information along with the hash. In other words, in addition to what we do now, MD5:PASSWORD_HASH, the developer would optionally append one or more sequence of: password character|password character position.

Below is an example of what the resulting password hash entry could look like. The hash length is fixed so we know the t at the end is the first password data character, the integer after is the position of that character within the source password, the : is a delimiter for the limited password data sequence.

MD5:3bcc3177cb5efa4d7341d42af8f06418**t2:%7**

When a user attempts to authenticate, you validate the hash per usual then additionally validate that those randomly saved password characters are indeed present in the password at the corresponding positions. This would largely nuke the utility of rainbow tables since exploiting weak hashes is now coupled with the need to in essence know something about the source password.

Of course the password data does reveal some bit of information about the password for anyone who has access to the database, but with a strong enough password this should be a non-issue.

Now rubbing my Darwin beard as I wait for the take down :)

[joakime]

Then I would recommend you wait until Chrome and Firefox support the "algorithm" parameter on digest auth.

See:

• https://datatracker.ietf.org/doc/html/rfc7616
• https://bugzilla.mozilla.org/show_bug.cgi?id=472823
• https://issues.chromium.org/issues/40054241

Firefox 93 added support for SHA-256 for the WWW-Authenticate digest challenge, but falls back to MD5 on preemptive auth. Chromium / Chrome still only does MD5. Much of the conversation around the "algorithm" parameter is about how to make it available everywhere (even javascript).

Currently the only support being implemented for this "algorithm" parameter is for the HTTP auth APIs. (the modal popup for user + password you see in some browsers)

Note all of the other/various HTTP Clients will also need to update before you can use this "algorithm" parameter universally. Eg: if you add a REST API, the REST client libraries wouldn't be able to use this "algorithm" parameter until they also update. Your database will have to support MD5 as a fallback, for every single user you have, for quite a long time (years).

Alternative, don't use HTTP authentication.

If you have a web page with a form based login, then this isn't using the HTTP Auth specs. You are free to pass anything you want via those forms, go hog wild, just remember that if you want to hash the password, you'll have to do that entirely in javascript on the client side before the form is even submitted. Remember to disable the form if javascript isn't enabled, don't want people to submit non-hashed passwords. If you want to go this route, then look at Servlet security, the Form authentication type, and you'll want a LoginService that can deal with those SHA-256 hashed passwords.

This will be custom (or you can look for someone that has already done this with JAAS or JASPI layers).

In the end, you wont use UnixCrypt or Password classes if you don't use HTTP authentication.

[joakime]

Oh!, and be sure to test the various browsers you want to work in to see if they even support SHA-256 in Javascript.

Note that some browsers won't allow you to access the SHA-256 libs in javascript unless the connection is already secured with TLS (a strange prerequisite, presumably done to encourage proper security behaviors).

[Devmond]

@joakime

I just realized some 7 years ago, I asked the same question regarding password hashing and got nowhere: https://www.eclipse.org/lists/jetty-dev/msg02883.html

I am not sure how your suggestions address my concern.

• I have a webapp that runs on Jetty.

• When a user creates an account, I create a hash of the password and store that in the database.

• When the user attempts to login, the default jetty LoginService performs a hash of the submitted password and checks it against the hash that was saved during account creation. Jetty relies on the MD5: or CRYPT: prefix to determine which hash to apply to the supplied password.

• The problem is that Jetty ships with a rather substandard set of options for password hashing, ie MD5 and CRYPT, not even with support for salting. Meaning any webapp that relies on Jetty's default LoginService options is effectively not secure.

There are alternatives out there, such as rolling your own JAAS module as you alluded to or using something like Spring security.

It would be nice if Jetty out of the box provided suitable support to current approaches for password security.

[joakime]

If it is using DIGEST HTTP auth then the hash is sent from the browser (in the form "user:realm:password"), there's no need for a LoginService, as there's no username + password pair sent. The DigestAuthenticator kicks in in this scenario.

This is what the Server sends on the Digest auth challenge response header (I've added new-lines to make it easier to read, these newlines wouldn't actually be there)

WWW-Authenticate: Digest
    realm="example-realm",
    qop="auth, auth-int",
    nonce="7ypf/xlj9XXwfDPEoM4URrv/xwf94BcCAzFZH4GiTo0v"

The client then sends its request with the following header (again, newlines are not there in a real request header)

Authorization: Digest username="Mufasa",
    realm="example-realm",
    uri="/dir/index.html",
    nonce="7ypf/xlj9XXwfDPEoM4URrv/xwf94BcCAzFZH4GiTo0v",
    nc=1,
    qop=auth,
    response="8ca523f5e9506fed4657c9700eebdbec"

The response field is the MD5 hash of user:realm:password When the server side receives it, the hashing of the password is basically done. The UnixCrypt / Password classes are not used with this kind of authentication.

The Jetty DigestAuthenticator uses java.security.MessageDigest instead to verify the submitted Authorization header per the https://datatracker.ietf.org/doc/html/rfc7616 rules.

[Devmond]

@joakime

Maybe in another 7 years I'll forget we had this conversion and start a new :)

The webapp as with virtually all web apps uses form based authenticate with a session, not digest.

I'll go dig into Spring and see if there are solutions to be found there.

Jetty should address this issue.

Thanks for your help.

[joakime]

• The problem is that Jetty ships with a rather substandard set of options for password hashing, ie MD5 and CRYPT, not even with support for salting. Meaning any webapp that relies on Jetty's default LoginService options is effectively not secure.

Jetty cannot invent techniques or technologies that do not exist on the HTTP protocol. We support what the various HTTP specs support. We have 2 options with the default HTTP specs: plaintext or Digest, and digest only supports MD5 (no salt) at this point in time.

While there is work being made to the specs to support more hashing algorithms, that support is lacking in the general internet at this point in time. (eg: no support from Chromium / Chrome yet) Even if we added support on the Jetty side for RFC7616's new "algorithm" parameter to allow SHA-256, there's nothing we can do about having Chromium / Chrome not supporting it. Also the new RFC7616 spec also has no salt, like you are asking about. Go ahead and read it https://httpwg.org/specs/rfc7616.html. Finally MD5 support is still a requirement, it cannot be eliminated. As you still need to support deficient clients (like Chromium / Chrome), which means you still have MD5 entries for every single user in your database (something you have stated repeatedly that you do not want).

There are alternatives out there, such as rolling your own JAAS module as you alluded to or using something like Spring security.

Spring security is an application level security layer. Well past HTTP, and the Servlet security specs. In application level security you can do whatever you want to do, as you are not beholden to the HTTP spec rules, or Servlet rules, or what the Browser can support, etc.

It would be nice if Jetty out of the box provided suitable support to current approaches for password security.

Jetty security has 2 main integration points.

1. HTTP protocol authentication / authorization.
2. Servlet spec security / constraints. (which also uses HTTP protocol security, with an optional Form login)

With HTTP protocol authentication you have options like Plaintext / Digest. And then there's HTTP authentication features like OAuth, LDAP, SPNEGO, etc.

With Servlet security you have all of the HTTP protocol techniques, along with constraints, and optional form security. Form security is an old school and very rudimentary application authentication method built into the Servlet spec. See https://docs.oracle.com/cd/E19226-01/820-7627/bncby/index.html Form security will send the username and password in plaintext to the server via a normal POST request with the username/password pair in either the query parameters or request body as either application/x-www-form-urlencoded or multipart/form-data. Servlet form security is rarely used in the modern era.