pwnedkeys - Githubissues

schuetzm commented 5 years ago

Matt Palmer just anounced a public database of compromised keys:

https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/mSXCMmw8x8M https://pwnedkeys.com/

Might be useful to integrate this.

drwetter commented 5 years ago

Thx!

That wouldn't be difficult to set up. I am only wondering why the returned JSON payload value is base64 encoded :-( But that's no real issue.

From the practical perspective I am wondering how many keys are in his DB and it's hard to understand why he doesn't include RSA keys < 2048 bit. Yeah, it's weak, but still...

drwetter commented 5 years ago

PoC:

curl -s https://v1.pwnedkeys.com/9e03b56749abe821a6f5299d6f634b35404975f0552eb3347bf3adfad9af1109 | sed -e 's/.*payload":"//g' -e 's/","protected".*$//' | openssl base64 -d -A

drwetter commented 5 years ago

... and what is weird, if I try with the certificate from testssl.sh

prompt% curl -s https://v1.pwnedkeys.com/ca0f134494ea7cad519a95e2595714d82fac2a1bb9aa04476b7590cf1886c38c
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
prompt%

So, the JSON has base64 encoded cleartext values, keys not in his DB return a server error and he refuses to list small keys. Thus at the moment I would rather say, let's wait until this matures. Until then this will carry a wontfix.

@schuetzm : if you happen to see a change, please let me know.

dcooper16 commented 5 years ago

The behavior is consistent with the information at https://pwnedkeys.com/search.html:

Make a request to https://v1.pwnedkeys.com/<fingerprint>. If the response to the request is a 200 OK, then the key appears in the database, while a 404 response indicates that the key doesn’t appear in the pwnedkeys database.

So, one could just do the query as:

curl --head https://www.google.com/ca0f134494ea7cad519a95e2595714d82fac2a1bb9aa04476b7590cf1886c38c

and the search the response for either "200 OK" or "404 Not Found". There doesn't seem to be any need to look at the JSON.

The value does seem to be limited, though. Presumably the key is bad if the server responds "200 OK", but a response of "404 Not Found" doesn't really provide much assurance.

Could it be added as a check that is performed if $PHONE_OUT is true, with the results being marked as experimental?

drwetter commented 5 years ago

Hi David,

and the search the response for either "200 OK" or "404 Not Found". There doesn't seem to be any need to look at the JSON.

actually I read the 200 too but it seems I was stuck then scratching my head what the JSON with the base64 values are about. ;-(

The 200 makes my impression slightly better.

The current value of this services is not clear to me. Momentarily I would not expect a big service like ihavebenpwned,com but I couldn't find any hint how many keys he has. Is it 3, 30, 300? 3000, 30000?

Also private keys expire. Pwned passwords still don't expire as they will be reused (actually "some" even recommend to check password against those from breaches: https://www.passwordping.com/nist-csf-800-63b-passwords/ ;) )

Could it be added as a check that is performed if $PHONE_OUT is true, with the results being marked as experimental?

That was my first idea too but the question is still whether it's worth it. Until the numbers of keys retrieved are not clear including their expiration date I am yet not convinced we should do it.

Maybe in the future this takes off?

mpalmer commented 5 years ago

Howdy, thought I'd respond to a few things raised in this issue, since I came across it.

I am only wondering why the returned JSON payload value is base64 encoded

The easy answer is, "because that's what JOSE requires" (of which the JSON Web Signatures that pwnedkeys uses for compromise attestation is a component). As to why JOSE requires all content to be base64-encoded, it's to avoid reformatting causing signature validation to spuriously fail. JSON canonicalisation is hard.

On the larger issue of "why the dickens did you inflict JWS on us, you MONSTER?!?", there's a FAQ for that. :grinning:

If you don't care about cryptographic verification that the key is, indeed, compromised, you can ignore the response body and just check the HTTP response code. It's delivered over HTTPS, so it's as trustworthy as anything else on the Internet. (cough)

From the practical perspective I am wondering how many keys are in his DB and it's hard to understand why he doesn't include RSA keys < 2048 bit.

When I started the project, I was focused on the publicly-trusted SSL/TLS certificate space, for which RSA keys less than 2048 bits in length have been prohibited for some time. Since then, I've been convinced that there is sufficient value in keeping track of smaller keys, so now RSA keys down to 1024 bits are included in the database.

As to how many keys are in the DB:

pwnedkeys=> select now(),count(id) from pwnedkeys;
              now              |  count  
-------------------------------+---------
 2019-05-21 00:51:39.435766+00 | 1129138
(1 row)

Most of those are so-called "Debian Weak Keys" that I've generated, but a growing number of them are gathered from other sources, and I'm working on more scraping sources as time permits -- and I'm always open to submissions.

keys not in his DB return a server error [...] a response of "404 Not Found" doesn't really provide much assurance.

(Nitpickers' corner: 404 is a request error, not a server error)

There's no useful response body I could think of to return, and using HTTP response codes to indicate non-existence allows for quicker and easier processing, so I went with it. I can't provide a cryptographic attestation of non-existence that's significantly more trustworthy than the 404 provided over HTTPS (assuming you're doing cert validation, etc). Returning a 200 and then some sort of "nope, not found" JSON just increases the required complexity for clients.

Also private keys expire.

Certificates expire. Private keys do not.

As to whether it's "worth it", I'll leave that up to others to decide, as I am, unsurprisingly, very, very biased.

drwetter commented 5 years ago

Hi @mpalmer ,

thanks for commenting!

I am only wondering why the returned JSON payload value is base64 encoded

The easy answer is, "because that's what JOSE requires" (of which the JSON Web Signatures that pwnedkeys uses for compromise attestation is a component).

Ah, ok. Didn't spot / remember that while curling by that time.

On the larger issue of "why the dickens did you inflict JWS on us, you MONSTER?!?",

I am tempted to ask this next time customers who don't understand what JWS cannot do. ;-)

As to how many keys are in the DB: [..]

thanks!

Actually I changed my mind, i.e. it cannot hurt to do this using --phone-out. I stumbled over a known key in a consulting project (which I still have to submit).

Also private keys expire.
Certificates expire. Private keys do not.

excuse me being sloppy. To me it is a common task when creating a new certificate request also to create a new private key. What I didn't take into account was that other's mileage may vary.

Was there any problem yesterday? Got only 502s.

Cheers, Dirk

dcooper16 commented 5 years ago

I had written a check for this some time ago, but didn't submit it as a PR since the issue was marked as "wontfix." However, since the issue is being reconsidered, I have submitted my code as #1274.

a response of "404 Not Found" doesn't really provide much assurance.
There's no useful response body I could think of to return, and using HTTP response codes to indicate non-existence allows for quicker and easier processing, so I went with it. I can't provide a cryptographic attestation of non-existence that's significantly more trustworthy than the 404 provided over HTTPS (assuming you're doing cert validation, etc). Returning a 200 and then some sort of "nope, not found" JSON just increases the required complexity for clients.

It's been five months since I made this comment, but I don't believe I was suggesting a possibility that a "404 Not Found" response might be provided even if the key is really in the database. I believe that I meant that a response of "404 Not Found" just indicates that the key does not appear in the database, and at least without more information about the database, simply knowing that a key does not appear in the database doesn't much assurance that the key isn't bad.

mpalmer commented 5 years ago

without more information about the database, simply knowing that a key does not appear in the database doesn't much assurance that the key isn't bad.

Aaah, yes, the old "can't prove a negative" problem. Without managing every private key yourself, generating it in a "known good" environment with suitable randomness and so on (ask Debian users circa 2008, or Infineon TPM users about how to validate that one), you really can never be sure that a key isn't bad. The best you can do is perform as many sanity checks as you can (of which pwnedkeys is one; ROCA, Debian weak key, appropriate exponent, and a bunch of others also need to be considered) and then cross your fingers.

Was there any problem yesterday? Got only 502s.

Yes, moving the API to a new system, things did not go 100% well. Should be all good now, though.

drwetter commented 5 years ago

Thanks for the input @mpalmer and thanks @dcooper16 for the PR #1274

drwetter / testssl.sh

pwnedkeys #1176