search

keybase / keybase-issues

A single repo for managing publicly recognized issues with the keybase client, installer, and website.
899 stars 37 forks source link

The Keybase web verification is absolutely broken and does not handle redirects. #1283

Open msjyoo opened 9 years ago

msjyoo commented 9 years ago

Hello,

It seems that keybase wants a 200 response to a HTTP(S) request in order to prove that a host is up. Instead, my host returns a 301 redirect on HTTP requests - a redirect to HTTPS.

This a very real-world scenario and is completely in line with the current web standards. It appears that Keybase does not follow the web standards, but instead uses a very hacky mechanism to check for whether a website is online.

In addition, the error message Keybase gives when a web verification fails is extremely vague. One cannot tell the difference of "I am not going to return true because I hate you" and "There is a problem with the website's certificate".

On attempt to verify my website by the CLI, I get a

error: Failed to find a connection to www.michael.yoo.id.au (error #229)

This is not only me, but many, many others who are reporting the same issue: https://github.com/keybase/keybase-issues/issues/962 https://github.com/keybase/keybase-issues/issues/677

The web interface has the same issue.

Keybase must change it's mechanism for checking whether a website is online. The current system is broken and does not cover all scenarios. Keybase should also follow redirect headers, and output a better error message for the users to better understand whether the problem is of their own fault, or is a bug.

/end wall of text

But seriously, can you fix this bug and use better error messages? Thanks!

ChristopherBurg commented 9 years ago

I'm have the same issue.

The HTTP connection on my site uses a 301 redirect to the HTTPS connection. Keybase has been unable to verify my site even though I am able to view the keybase.txt file without issue in Firefox, Chrome, and Safari.

From the keybase command line tool (v0.7.3) I receive the following error:

Check https://christopherburg.com/.well-known/keybase.txt OR https://christopherburg.com/keybase.txt now? [Y/n] Y

warn: Didn't find the posted proof.

JamborJan commented 9 years ago

Same here.

301 redirect on HTTP requests and "could not query mydomain.com. Are you running a site there?" when I try to "Prove yourself, webmaster!"

malgorithms commented 9 years ago

hey all, just to be clear, we do folllow 301 http-https redirects, and we have for a long time. When this fails there's something else wrong. Error reporting in the future will improve, but don't forget the site is in alpha, and we were pretty clear up front that the site is in active development.

@sekjun9878 you don't seem ok with the alpha (feature incomplete) state of the site. If you want to wait until we're in beta, I totally understand.

As for the specifics here:

@ChristopherBurg - your site has an infinite redirect loop when visiting https://christopherburg.com/.well-known/keybase.txt which is what keybase tries first. Of course a better error message will come from us eventually. We stop trying your site when we detect something like that. (it's going keybase.txt -> 404 -> 404 -> 404 -> 404 ....). You can confirm this with firefox or just with some curl -I statements. I think you need to redirect to "/404" instead of "404", if that's your goal.

@JamborJan - I like that your site has a big link to your keybase profile. Anyway, I think the problem with yours is that you're missing an intermediate certificate so SSL isn't configured right. Check it out in firefox:

image

Chrome lets this slide because it comes bundled with more intermediate certificates. But this is something you should fix so https works for people in all browsers.

@sekjun9878 - I get error 500's on your site for both places a keybase.txt file can sit.

> curl https://www.michael.yoo.id.au/keybase.txt

<!DOCTYPE html><!-- "' --></script></style></noscript></xmp>
<meta charset="utf-8">
<meta name=robots content=noindex>
<meta name=generator content="Tracy">
<title>Server Error</title>

<div id="tracy-error-body">
<style>
    body { color: #333; background: white; width: 500px; margin: 100px auto }
    h1 { font: bold 47px/1.5 sans-serif; margin: .6em 0 }
    p { font: 21px/1.5 Georgia,serif; margin: 1.5em 0 }
    small { font-size: 70%; color: gray }
</style>

<h1>Server Error</h1>

<p>We're sorry! The server encountered an internal error and
was unable to complete your request. Please try again later.</p>

<p><small>error 500</small></p>
<div>

<script>
document.documentElement.innerHTML = '<title>Server Error<\/title>' +
    document.getElementById('tracy-error-body').innerHTML;
</script>
malgorithms commented 9 years ago

@JamborJan - more specific information: whoever provided you with your SSL certificate should also provide you with this intermediate certificate:

image

msjyoo commented 9 years ago

@malgorithms First of all, thanks for replying!

I understand that the site and the software is in a very early stage of development, which is why I am here to give feedback to improve keybase!

Regarding the specific error 500 you pointed out, that is due to my CMS being configured to give 500 for paths not found. I could not generate the data to make a keybase.txt in the first place, as both the CLI and the website gave errors when trying to generate the verification text.

I have put in fake data in place so you can now confirm that this is not a problem with my server configuration (hopefully).

C:\Users\Michael>curl -i https://www.michael.yoo.id.au/keybase.txt
HTTP/1.1 200 OK
Content-Type: text/plain
Last-Modified: Fri, 16 Jan 2015 15:30:36 GMT
Accept-Ranges: bytes
ETag: W/"93af360a131d01:0"
Server: Microsoft-IIS/8.5
Date: Fri, 16 Jan 2015 15:32:55 GMT
Content-Length: 30

keybase file test /keybase.txt
C:\Users\Michael>curl -i https://www.michael.yoo.id.au/.well-known/keybase.txt
HTTP/1.1 200 OK
Content-Type: text/plain
Last-Modified: Fri, 16 Jan 2015 15:30:30 GMT
Accept-Ranges: bytes
ETag: W/"3426345ca131d01:0"
Server: Microsoft-IIS/8.5
Date: Fri, 16 Jan 2015 15:33:23 GMT
Content-Length: 41

keybase file test .well-known/keybase.txt

Note the lack of https here, and the redirect.

C:\Users\Michael>curl -i http://www.michael.yoo.id.au/.well-known/keybase.txt
HTTP/1.1 302 Redirect
Content-Type: text/html; charset=UTF-8
Location: https://www.michael.yoo.id.au/.well-known/keybase.txt
Server: Microsoft-IIS/8.5
Date: Fri, 16 Jan 2015 15:33:46 GMT
Content-Length: 176

<head><title>Document Moved</title></head>
<body><h1>Object Moved</h1>This document may be found <a HREF="https://www.michael.yoo.id.au/.well-known/keybase.txt">here</a></body>

You can see the error occurring when trying to generate the proof data:

C:\Users\Michael>keybase version
keybase (keybase.io CLI) v0.7.3
- node.js v0.10.33
- gpg (GnuPG) 2.0.26 (Gpg4win 2.2.3)
- libgcrypt 1.6.2
Identifies as: 'keybase.io node.js client v0.7.3 win32'
C:\Users\Michael>keybase prove https
info: Updated file: C:\Users\Michael\AppData\Roaming\keybase\session.json
Hostname to check: www.michael.yoo.id.au
error: Failed to find a connection to www.michael.yoo.id.au (error #229)
C:\Users\Michael>keybase prove https
Hostname to check: https://www.michael.yoo.id.au
error: Failed to find a connection to www.michael.yoo.id.au (error #229)
ChristopherBurg commented 9 years ago

Wow, I feel dumb. Yeah, had a botched redirect for 404 errors in my config file. I really should have checked more closely to see what was going on.

Thanks for the feedback @malgorithms.

malgorithms commented 9 years ago

@sekjun9878 - thanks for the info. I see what you mean now.

It appears your server doesn't like something about Node making https requests to it. Consider this pretty boilerplate code:

var https = require('https');

var options = {
  hostname: 'www.michael.yoo.id.au',
  port: 443,
  path: '/',
  method: 'GET'
};

var req = https.request(options, function(res) {
  console.log("" + options.hostname + " res.statusCode=" + res.statusCode);
  return process.exit(0);
});

req.end();

When I run this, your site replies with a 500. (Other sites reply with 200.) And when I make a request to your URL via cURL, I get a 200 reply. So it's Node + your site = your site giving a 500.

Is there something about your server that isn't open to getting requests from certain user agents?

JamborJan commented 9 years ago

Thanks for checking @malgorithms. That message seems a bit odd to me as I have tested it on several browsers including firefox and various devices and other clients. I have never seen it. I double checked the settings and it should all be ok now. I had forgotten to add the Intermediate CA. Anyway the "Prove yourself, webmaster!" process is not working anyway.

Can you please check if you still get the error message?

What else can I do?

Thx a lot!

msjyoo commented 9 years ago

@malgorithms Hmm.. it seems that you are right. Modifying the node.js to contain a User-Agent header fixed the problem (at least in the boilerplate code.) However, I think that this is to do with node.js specifically as curl does not inhibit this issue when using the same request headers. (And I am not in a position to debug NodeJS issues as I do not use the language unfortunately.)

I used this example to as the node.js boilerplate code for a more verbose output: http://nodejs.org/api/https.html#https_https_request_options_callback.

Output without User-Agent:

C:\Users\Michael\Desktop>node test.js
statusCode:  500
headers:  { 'content-type': 'text/html; charset=UTF-8',
  server: 'Microsoft-IIS/8.5',
  'x-powered-by': 'PHP/5.6.0',
  date: 'Sun, 18 Jan 2015 07:17:22 GMT',
  'content-length': '788' }
<!DOCTYPE html><!-- "' --></script></style></noscript></xmp>
<meta charset="utf-8">
<meta name=robots content=noindex>
<meta name=generator content="Tracy">
<title>Server Error</title>
...body content truncated by me

Output with User-Agent by modifying headers like so:

var options = {
  hostname: 'www.michael.yoo.id.au',
  port: 443,
  path: '/',
  method: 'GET',
  headers: {
      'User-Agent': 'NodeJS/1.0'
  }
};
C:\Users\Michael\Desktop>node test.js
statusCode:  200
headers:  { 'content-length': '6371',
  'content-type': 'text/html;charset=UTF-8',
  server: 'Microsoft-IIS/8.5',
  'x-powered-by': 'PHP/5.6.0',
  date: 'Sun, 18 Jan 2015 07:19:05 GMT',
  connection: 'close' }
<!DOCTYPE html>
<!--

    Hello!
...more body output truncated by me

Here is curl's verbose output without any user agent headers, which shows that this is not a configuration problem with my server blocking empty user agents:

C:\Users\Michael\Desktop>curl -v -H "User-Agent:" https://www.michael.yoo.id.au/.well-known/keybase.txt
* Adding handle: conn: 0x78ea10
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x78ea10) send_pipe: 1, recv_pipe: 0
* About to connect() to www.michael.yoo.id.au port 443 (#0)
*   Trying 104.36.87.196...
* Connected to www.michael.yoo.id.au (104.36.87.196) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: C:\Program Files (x86)\Git\bin\curl-ca-bundle.crt
  CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using AES256-SHA
* Server certificate:
*        subject: OU=Domain Control Validated; OU=PositiveSSL; CN=www.michael.yoo.id.au
*        start date: 2014-09-19 00:00:00 GMT
*        expire date: 2016-09-18 23:59:59 GMT
*        subjectAltName: www.michael.yoo.id.au matched
*        issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Domain Validation Secure Server CA
*        SSL certificate verify ok.
> GET /.well-known/keybase.txt HTTP/1.1
> Host: www.michael.yoo.id.au
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Last-Modified: Fri, 16 Jan 2015 15:30:30 GMT
< Accept-Ranges: bytes
< ETag: "3426345ca131d01:0"
* Server Microsoft-IIS/8.5 is not blacklisted
< Server: Microsoft-IIS/8.5
< Date: Sun, 18 Jan 2015 07:10:15 GMT
< Content-Length: 41
<
keybase file test .well-known/keybase.txt* Connection #0 to host www.michael.yoo.id.au left intact

You can see here that curl works fine without any User-Agent headers:

> GET /.well-known/keybase.txt HTTP/1.1
> Host: www.michael.yoo.id.au
> Accept: */*
>
< HTTP/1.1 200 OK
msjyoo commented 9 years ago

@JamborJan I think the issue with your website is that your webserver is not sending the full trust chain for clients with only root CAs installed. Here is the trust chain of your website as reported by Qualys SSLLabs: www_jambor_pro_tls_certification_path

To contrast, my website and keybase's website both send the full certificate chain: www_michael_yoo_id_au_tls_certification_path

www_keybase_io_tls_certification_path

When attempting to use the boilerplate code to access your website:

C:\Users\Michael\Desktop>node test.js
[Error: UNABLE_TO_VERIFY_LEAF_SIGNATURE]

Like @malgorithms has said, this issue is not seen in most browsers because they have most intermediate certificates pre-installed, hence the "Extra Download". However, this is not a requirement and it appears that keybase (nodejs) only stores the root CAs, therefore you need to provide the intermediate certificate in your web server for full compatibility. (Source: serverfault.com link below.)

You can test your website's SSL again here: https://www.ssllabs.com/ssltest/analyze.html?d=jambor.pro&hideResults=on and click on CLEAR CACHE.

You can check this link out for a possible solution to your problem: http://serverfault.com/questions/628062/what-is-wrong-with-my-ssl-trust-chain

This might not be 100% correct, and I'm not an expert on this matter but hopefully this could be of help to you!

JamborJan commented 9 years ago

@sekjun9878 awesome help thanks!

I bundled now the certificates right and its working.

Just in case someone also gets a message like this after having bundled the crt files with cat:

# nginx -s reload
nginx: [emerg] PEM_read_bio_X509_AUX("/etc/nginx/www.jambor.pro_chained.crt") failed (SSL: error:0906D066:PEM routines:PEM_read_bio:bad end line)

Check your crt file and you will see for sure that the end of one certificate and the end of the next are in one line which is causing the issue:

Wrong

-----END CERTIFICATE----------BEGIN CERTIFICATE-----

Right

-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----

As I'm serving several pages with different certificates from one IP I still have a "No SNI 2" issue but thats not solvable. Result on https://www.ssllabs.com/ssltest is now A+ :-)

msjyoo commented 9 years ago

@malgorithms Any update on the issue?

msjyoo commented 9 years ago

@JamborJan Glad I could be of help :)

AntoineVe commented 9 years ago

Thanks to this issue, I can know prove my website with https. I've to add all the certificates to my crt ... Hints : to find cert, you can use https://www.tbs-certificates.co.uk/FAQ/en/search

joostrijneveld commented 9 years ago

This issue helped me resolve an issue where Keybase was not verifying my proof as well. As I had recently switched to HTTPS-redirects, I figured it would be related; in the end it was a missing intermediate CA certificate.

While I fully agree to the current behaviour on this aspect, here's another one for more verbose error messages ;-)

virtualdxs commented 7 years ago

I'm having the same problem with error 229 on https://box.k7dxs.xyz/. The weird thing is it worked previously.

sid3windr commented 7 years ago

+1 for better errors, here, too, I was thrown off by the 302 being the issue while in fact my older Apache 2.2 needed the separate chain.pem entry for LetsEncrypt as well (vs my 2.4s being ok with fullchain).