Danbooru web server: security and performance tweaks

[reiyasona]

Web server tests used + results:

• Qualys SSL Labs - SSL server test -> archived result
• CryptCheck - HTTPS test
• High-Tech Bridge - SSL/TLS server test
• Observatory by Mozilla
• securityheaders.io -> archived result
• Webbkoll
• KeyCDN - HTTP/2 test -> archived result

  SSL/TLS:

• [x] The Danbooru web server is vulnerable to the OpenSSL Padding Oracle vulnerability (CVE-2016-2107) and can therefore be considered insecure. Fix: OpenSSL 1.0.2 users should upgrade to at least 1.0.2h and OpenSSL 1.0.1 users should upgrade to at least 1.0.1t.
• [x] The web server server accepts the broken RC4 cipher (only with older protocol versions) in the following cipher suites: TLS_ECDHE_RSA_WITH_RC4_128_SHA & TLS_RSA_WITH_RC4_128_SHA. This is a minor issue, but in this day and age RC4 should be banned from the server configuration. An exception for this rule could be justified, if there were any important old clients accessing Danbooru over RC4. Fix: The Mozilla SSL Configuration Generator recommends this configuration for nginx 1.6.2 + OpenSSL 1.0.1t. This configuration will also guarantee Forward Secrecy with most modern clients. Warning: If the server admin wants to use HSTS, then he must be sure that he will be supporting HTTPS on whole site for a long period of time. HSTS might be overkill (also because Danbooru can still be accessed via plain HTTP :wink:).

• [x] Enable TLS Session Resumption for better performance and disable insecure Session Tickets.

  ssl_session_timeout 4h;
  ssl_session_cache shared:SSL:20m;
  ssl_session_tickets off;

  (SSL/TLS nginx configuration overview - slightly modified)

• [x] The intermediate certificate GeoTrust Global CA (SHA1 fingerprint: 7359755c6df9a0abc3060bce369564c8ec4542a3) signed by the Equifax root certificate (SHA1 fingerprint: d232­09ad­23d3­1423­2174­e40d­7f9d­6213­9786­633a) has a weak signature (SHA1withRSA). Furthermore, the Equifax root certificate was revoked by its certification authority because it uses a weak 1024 bit long RSA key. Fix: Remove the GeoTrust Global CA certificate (SHA1 fingerprint: 7359755c6df9a0abc3060bce369564c8ec4542a3) from the certificate chain. Afterwards the certificate chain should look like this. Possibly related: https://github.com/r888888888/danbooru/issues/2118

• [x] Enable OCSP Stapling to improve user privacy.

  ssl_stapling on;
  ssl_stapling_verify on;
  ssl_trusted_certificate /path/to/root_CA_cert_plus_intermediates.pem;
  resolver 8.8.8.8 8.8.4.4;

  The content of Danbooru's "root_CA_cert_plus_intermediates.pem" file should look like this. Safebooru's should look like this. (SSL/TLS nginx configuration overview)

  HTTP:

• [x] Update nginx to at least 1.9.5 to benefit from the HTTP/2 performance boost on modern clients.

  server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
  ...

  (SSL/TLS nginx configuration overview)

[Type-kun]

No HSTS please. It's easy to forget about it, but there are local networks where HTTPS is blocked as a whole; no reason to deny those people access to danbooru.

[r888888888]

• I've upgraded OpenSSL.
• I've removed the broken ciphers from Nginx.
• I would like to hold off on upgrading nginx until I decide to do a dist-upgrade. Past experiences with manually installing packages have not been pleasant.
• I assume fixing the intermediate cert means getting a new cert from somewhere else. I don't 100% understand how they work and if the Equifax cert is tied to the RapidSSL cert.

[r888888888]

Actually the SSL cert issue is fixed now, on both danbooru and safebooru.

[reiyasona]

I assume fixing the intermediate cert means getting a new cert from somewhere else. I don't 100% understand how they work and if the Equifax cert is tied to the RapidSSL cert.

The Equifax cert is not tied to the RapidSSL cert.

Actually the SSL cert issue is fixed now, on both danbooru and safebooru.

I just updated my list and included the full certificate chain for clarification. But you already solved this on your own. Great! :laughing:

I also added OCSP Stapling as a new feature request.

[reiyasona]

Can we reopen this issue? TLS Session Resumption and OCSP Stapling are pretty nice features.

@r888888888 I have two questions regarding Session Resumption fine tuning:

• Can you tell me how many TLS handshakes Danbooru performs on a daily basis?
• Where can I find detailed information on the Danbooru server hardware?

[r888888888]

I've added session resumption and ocsp stapling.

Not sure how I'd count the number of TLS handshakes that happen.

[reiyasona]

I've added session resumption and ocsp stapling.

Great!

Not sure how I'd count the number of TLS handshakes that happen.

To accomplish this you would have to define a custom log_format in the http context (e.g. /etc/nginx/nginx.conf) and add $ssl_cipher to this configuration. Afterwards, it's time for some web server log file analysis with GoAccess (or similar). However, I think this might be a bit too complicated.

Maybe we should keep it simple and just look at the unique visitor count of Danbooru. I noticed that you are running Google Analytics and New Relic.

In addition, you could carry out some very primitive spot checks with:

netstat -ntu | grep -w 80 | grep -v LISTEN | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | grep -v 127.0.0.1 | wc -l
netstat -ntu | grep -w 443 | grep -v LISTEN | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | grep -v 127.0.0.1 | wc -l

This should output the total amount of all active IP’s that are currently connected to Danbooru's web server via port 80 and port 443.

A more straightforward approach is stress testing. You could set the sub-domain danbooru.donmai.us "HTTPS only" for one day and analyze the server's performance. You should also keep track of the visitor count on this particular day (see Type-kun's comment above).

NGINX server config:

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    # Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
    return 301 https://$host$request_uri;
}

[reiyasona]

To achieve better compatibility with older clients, we should enable the following cipher suites and protocols:

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';
ssl_prefer_server_ciphers on;

[r888888888]

Why support tls v1? Seems like it has some clear weaknesses.

[kittey]

Every non-ancient browser and library supports TLS v1. Some browsers were rather late with introducing TLS v1.1 or 1.2 support:

Android browser: since Android 5 (November 2014) Chrome: since version 22 (around 2008) Firefox: since version 27 (February 2014) Internet Explorer: since version 11 (October 2013 / Windows 7 and up) Opera: since version 12.18 (February 2016) Safari: since OS X 10.9 (October 2013) Safari: since iOS 5 (around 2011)

If you have the newest version of whatever, you’ll be fine, but some systems either cannot be upgraded (especially mobile ones) or use extended support versions (Linux distributions, managed Windows environments like universities).

From what I see, most security sites recommend disabling TLS v1 (and v1.1) if you don’t need it, but it’s acceptable to have it enabled. Server testers like the one on ssllabs.com mark it as grey - neither good nor bad. New browsers use TLS v1.2 when available, so it’s only an issue for older browsers or if there’s a practical downgrade attack available, which it isn’t at the moment, AFAIK.

My personal feeling is that it’s still okay to keep TLS v1 around a bit longer.

[reiyasona]

TLSv1 has its weaknesses (e.g. AES CBC mode) and some implementations suffered from SSLv3 downgrade attacks, but most of the problems have already been fixed or mitigated.

Clients who address Danbooru over TLSv1 will usually use the cipher suit TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(NGINX calls it DHE-RSA-AES128-GCM-SHA256). This is still a very safe connection. It's even granting forward secrecy!

Be sure to include the ssl_prefer_server_ciphers on; part in your NGINX config. This will urge the clients to respect the cipher suit order presented by the server.

BTW: Why does NGINX serve the certificate of danbooru.donmai.us for sonohara.donmai.us and hijiribe.donmai.us (see post #116760)? Shouldn't we disable HTTPS (443) on these domains?

[kittey]

BTW: Why does NGINX serve the certificate of danbooru.donmai.us for sonohara.donmai.us and hijiribe.donmai.us (see post #116760)? Shouldn't we disable HTTPS (443) on these domains?

Danbooru only has those two webservers. danbooru.donmai.us is aliased to them, so they have to serve that certificate (if SNI is not used) and HTTPS has to be enabled.

Users shouldn’t be using hijiribe/sonohara directly anyway, so I’m all for letting them suffer. >:D

Maybe anyone using those domains should be redirected to the main domain, but that’s getting a bit off-topic here.

[reiyasona]

Danbooru only has those two webservers. danbooru.donmai.us is aliased to them, so they have to serve that certificate (if SNI is not used) and HTTPS has to be enabled.

Okay, I see. However, HTTPS on Danbooru works only in combination with browsers that are supporting SNI in the first place. All modern Browsers support SNI and still get SSL_ERROR_BAD_CERT_DOMAIN on hijiribe and sonohara. The web server does neither serve a certificate with a fitting subjectAltName field present nor a dedicated single-domain certificate (for hijiribe/sonohara).

Users shouldn’t be using hijiribe/sonohara directly anyway, so I’m all for letting them suffer. >:D

Maybe anyone using those domains should be redirected to the main domain, but that’s getting a bit off-topic here.

You are truly evil. :P

[r888888888]

I've made the SSL protocol changes.

[reiyasona]

I forgot to mention the weak default 1024 DH parameters.

• [x] Please generate strong 2048 bit key exchange parameters by executing:

  openssl dhparam 2048 -out dhparam.pem

  Afterwards you can link the dhparam.pem file in your NGINX config:

  # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
  ssl_dhparam /path/to/dhparam.pem;

[Type-kun]

Users shouldn’t be using hijiribe / sonohara directly anyway, so I’m all for letting them suffer. >:D

Yet subdomains show different speed somehow :3 When danbooru is staggering, hijiribe can be working fine.

Also, today I was able to access https with Opera 12.16, no problem so far. TLS v1 helped, I guess.

[r888888888]

I've gone ahead and upgraded nginx and enabled HTTP 2 support.

[xiaody]

I saw the HTTP/2 support notification on site, but it is not actually enabled for Chrome users. Seems due to the NPN/ALPN issue: https://www.nginx.com/blog/supporting-http2-google-chrome-users/

[reiyasona]

You're right, up-to-date versions of Chrome and Edge only support ALPN (the successor of NPN). The Chrome Devs decided to ditch NPN support earlier this year. The Danbooru web server (nginx/1.11.3) currently can not offer ALPN because the OS is still relying on OpenSSL 1.0.1. If you want to use HTTP/2 on Danbooru or any server running OpenSSL 1.0.1, you'll have to switch over to NPN compatible browsers like Firefox 35+ or Safari 9+ (both support ALPN + NPN). Another option would be to manually upgrade the OpenSSL packages of the server to 1.0.2. However, this is a really bad idea because there exist lots of other services that strongly depend on system shipped OpenSSL libraries. Let's just wait for the OpenSSL 1.0.2 packages of Debain Stretch to arrive.

[reiyasona]

@r888888888

Debian 9 "Stretch" is finally here and it comes with OpenSSL 1.1.0 and ALPN - i.e., HTTP/2 support for Chrome and Edge users.