SSL Public key Pinning support for SecurityContext

sandeepcmsm commented 5 years ago

Support for pinning base64 SHA-256 hashes as in HTTP Public Key Pinning (HPKP) or SHA-1 base64 hashes in SecurityContext. Support for Subject Public Key Info in X509Certificate. normally in native android and ios apps key pinning done similar like this https://square.github.io/okhttp/3.x/okhttp/okhttp3/CertificatePinner.html

rsilvr commented 5 years ago

I'm also in need of such a feature. Even certificate pinning I found that is only possible in dart to add a self-signed certificate as trusted (in the badCertificateCallback), but I couldn't find any way of validating myself in every call the certificate of the server I'm connecting to. In a MITM attack the default security context implementation wouldn't be enough.

ivnsch commented 5 years ago

@rsilvr Are you saying that certificate pinning doesn't work with arbitrary certificates? Can anyone confirm this? This would make this functionality pretty much useless.

sandeepcmsm commented 5 years ago

@i-schuetz factory SecurityContext({bool withTrustedRoots: false}); by setting withTrustedRoots to false will deny all request even if they are root trusted certificates and will fallback to badCertificateCallback. Then you can verify your self signed certificates.

rsilvr commented 5 years ago

@sandeepcmsm Thank you a lot for making this clear! I've tried something similar to this some months ago but I have no idea why it wasn't working before...

pythoneer commented 5 years ago

@i-schuetz what do you mean by "This would make this functionality pretty much useless." There is no functionality to pin a key at all. At least that i am aware of.

The only thing that comes close is to use SecurityContext({bool withTrustedRoots: false}); to make every certificate fail and use badCertificateCallback to check the public key, but this "disables" the validation of the certificate chain. So if you want to use public key pinning to avoid MITM attacks (by installing fake certificates on the device) i think this is useless. I am not 100% sure but i guess one can crate a certificate with the public key, it is just not valid – but that is not a problem for the attacker because you deactivated validation to check the public key.

All you can do currently is "first check the public key" with a webservice call and validation deactivated (a "pilot" call) and after that activate it (with withTrustedRoots: true) to make the "original" call. The problem here is, that an attacker knows it needs to present a certificate with the right public key (but not validateable) on the first "pilot" call and after that just a self signed certificate installed on the device (but with a wrong public key) on the second call. And this makes both approaches useless – i think.

The only "right" way is to either have a callback for all requests the gives us the Certificate (like in badCertificateCallback) but not just for the bad ones and we can check the public key by hand. Or even better, give the SecurityContext a list of public keys we want to allow when we have withTrustedRoots: true.

Because what we want is to validate AND check the public key on the same webservice call uninterrupted (roughly on the same connection).

mleonhard commented 5 years ago

I am not 100% sure but i guess one can crate a certificate with the public key, it is just not valid – but that is not a problem for the attacker because you deactivated validation to check the public key.

I think you missed a point. Public key cryptography works because the public and private keys are linked by math. If you encrypt with the public key, only someone holding the private key can decrypt. If an attacker makes a new certificate using the same public key, they cannot use it because they don't know the corresponding private key. SSL/TLS connection setup will fail.

Public-key cryptography, or asymmetric cryptography, is a cryptographic system that uses pairs of keys: public keys which may be disseminated widely, and private keys which are known only to the owner. The generation of such keys depends on cryptographic algorithms based on mathematical problems to produce one-way functions. Effective security only requires keeping the private key private; the public key can be openly distributed without compromising security.[1]

In such a system, any person can encrypt a message using the receiver's public key, but that encrypted message can only be decrypted with the receiver's private key.

https://en.wikipedia.org/wiki/Public-key_cryptography

So I believe one can implement certificate pinning just by checking the public key of the certificate. That's the only thing that matters for security.

mleonhard commented 5 years ago

I believe that certificate pinning is already possible without this feature request:

Create a root certificate and root private key.
Create a server certificate and private key.
Sign the server certificate with the root private key.
Create a certificate chain file by concatenating the signed server certificate and the root certificate files.
Configure your server with the server certificate and private key.
In your Dart code, create a SecurityContext that doesn't trust the OS's certificates: SecurityContext(withTrustedRoots: false);.
Call SecurityContext.setTrustedCertificates(rootCertificateBytes) to make Dart accept all certificates signed by the root private key.

It's possible that SecurityContext.setTrustedCertificates(selfSignedCertificate) would also work. I found nothing in the docs about this. If this is true, then we could eliminate some steps above.

HttpClient.badCertificateCallback is an alternative if you're doing only https. See https://stackoverflow.com/a/54838348

mleonhard commented 5 years ago

Key rotation reduces risk. When an attacker obtains an old server hard drive or backup file and gets an old server private key from it, they cannot impersonate the current server if the key has been rotated. Therefore always generate a new key when updating certificates. Configure the client to trust the old key and the new key. Wait for your users to update to the new version of the client. Then deploy the new key to your servers. Then you can remove the old key from the client.

The sole purpose of this feature request is to facilitate key pinning without rotation, a bad security practice. Please comment and explain if you disagree. Otherwise, let's close this feature request.

Below are several examples certificate pinning, supporting the good security practice of annual key rotation.

Create a self-signed certificate every year and have the clients trust only this year's and last year's certificates:

import 'dart:io'
    show
        BytesBuilder,
        File,
        HttpClient,
        HttpClientRequest,
        HttpClientResponse,
        HttpHeaders,
        HttpRequest,
        HttpServer,
        InternetAddress,
        Process,
        stderr,
        stdout,
        SecurityContext;
import 'dart:convert' show utf8;

Future<void> shellCommand(String command) async {
  print('Executing command $command');
  final Process process = await Process.start('sh', ['-c', command]);
  stdout.addStream(process.stdout);
  stderr.addStream(process.stderr);
  final int exitCode = await process.exitCode;
  if (exitCode != 0) {
    throw new Exception('Process exited with status $exitCode');
  }
}

void main() async {
  // Last year's certificate:
  await shellCommand(
      'openssl req -newkey rsa:2048 -passout pass:password -keyout privatekey2018.pem -subj "/CN=localhost" -days 731 -x509 -out certificate2018.pem');
  // This year's certificate:
  await shellCommand(
      'openssl req -newkey rsa:2048 -passout pass:password -keyout privatekey2019.pem -subj "/CN=localhost" -days 731 -x509 -out certificate2019.pem');

  final SecurityContext serverSecurityContext = new SecurityContext();
  serverSecurityContext.useCertificateChainBytes(
      await new File('certificate2019.pem').readAsBytes());
  serverSecurityContext.usePrivateKey('privatekey2019.pem',
      password: 'password');
  final HttpServer httpServer = await HttpServer.bindSecure(
      InternetAddress.loopbackIPv4, 0, serverSecurityContext);
  httpServer.listen((HttpRequest request) {
    request.response.write('body1');
    request.response.close();
  });
  print('Server listening at https://localhost:${httpServer.port}/');

  print('Making request.');
  final SecurityContext clientSecurityContext =
      new SecurityContext(withTrustedRoots: false);
  clientSecurityContext.setTrustedCertificatesBytes(
      await new File('certificate2018.pem').readAsBytes());
  clientSecurityContext.setTrustedCertificatesBytes(
      await new File('certificate2019.pem').readAsBytes());
  final HttpClient httpClient = new HttpClient(context: clientSecurityContext);
  final HttpClientRequest request = await httpClient.getUrl(Uri(
      scheme: 'https', host: 'localhost', port: httpServer.port, path: '/'));
  final HttpClientResponse response = await request.close();
  final List<int> bytes = await response.fold(new BytesBuilder(),
      (BytesBuilder bytesBuilder, List<int> bytes) {
    bytesBuilder.add(bytes);
    return bytesBuilder;
  }).then((BytesBuilder bytesBuilder) => bytesBuilder.takeBytes());
  final String contenType =
      response.headers.value(HttpHeaders.contentTypeHeader) ?? '';
  print('${response.statusCode} ${response.reasonPhrase} '
      'content-type="$contenType" body="${utf8.decode(bytes)}"');

  httpServer.close(force: true);
}

Create self-signed certificates and have the client trust only those certificates, and ignore the hostname in the request. This is useful when using Terraform to deploy the server to AWS Elastic Beanstalk. The server binary blob must contain the certificates, yet the server's hostname is not known until the deployment completes.

import 'dart:io'
    show
        BytesBuilder,
        File,
        HttpClient,
        HttpClientRequest,
        HttpClientResponse,
        HttpHeaders,
        HttpRequest,
        HttpServer,
        InternetAddress,
        Process,
        stderr,
        stdout,
        SecurityContext,
        X509Certificate;
import 'dart:convert' show utf8;

Future<void> shellCommand(String command) async {
  print('Executing command $command');
  final Process process = await Process.start('sh', ['-c', command]);
  stdout.addStream(process.stdout);
  stderr.addStream(process.stderr);
  final int exitCode = await process.exitCode;
  if (exitCode != 0) {
    throw new Exception('Process exited with status $exitCode');
  }
}

void main() async {
  // Last year's certificate:
  await shellCommand(
      'openssl req -newkey rsa:2048 -passout pass:password -keyout privatekey2018.pem -subj "/CN=localhost" -days 731 -x509 -out certificate2018.pem');
  // This year's certificate:
  await shellCommand(
      'openssl req -newkey rsa:2048 -passout pass:password -keyout privatekey2019.pem -subj "/CN=localhost" -days 731 -x509 -out certificate2019.pem');

  final SecurityContext serverSecurityContext = new SecurityContext();
  serverSecurityContext.useCertificateChainBytes(
      await new File('certificate2019.pem').readAsBytes());
  serverSecurityContext.usePrivateKey('privatekey2019.pem',
      password: 'password');
  final HttpServer httpServer = await HttpServer.bindSecure(
      InternetAddress.loopbackIPv4, 0, serverSecurityContext);
  httpServer.listen((HttpRequest request) {
    request.response.write('body1');
    request.response.close();
  });
  print('Server listening at https://localhost:${httpServer.port}/');

  print('Making request.');
  final SecurityContext clientSecurityContext =
      new SecurityContext(withTrustedRoots: false);
  final HttpClient httpClient = new HttpClient(context: clientSecurityContext);
  final List<String> certificatePemStrings = [
    await new File('certificate2018.pem').readAsString(),
    await new File('certificate2019.pem').readAsString()
  ];
  httpClient.badCertificateCallback =
      (X509Certificate cert, String host, int port) => certificatePemStrings
          .any((certificatePemString) => cert.pem == certificatePemString);
  final HttpClientRequest request = await httpClient.getUrl(Uri(
      scheme: 'https', host: 'localhost', port: httpServer.port, path: '/'));
  final HttpClientResponse response = await request.close();
  final List<int> bytes = await response.fold(new BytesBuilder(),
      (BytesBuilder bytesBuilder, List<int> bytes) {
    bytesBuilder.add(bytes);
    return bytesBuilder;
  }).then((BytesBuilder bytesBuilder) => bytesBuilder.takeBytes());
  final String contenType =
      response.headers.value(HttpHeaders.contentTypeHeader) ?? '';
  print('${response.statusCode} ${response.reasonPhrase} '
      'content-type="$contenType" body="${utf8.decode(bytes)}"');

  httpServer.close(force: true);
}

An alternative to pinning the public key is to use a certificate authority. Create the certificate authority files on a secure laptop and keep them on a removable drive in a safe. Whenever you need a new certificate, get the removable drive and generate and sign a new server certificate. Here's an example of the openssl commands to run and how to configure Dart to trust only certificates signed by your certificate authority:

import 'dart:io'
    show
        BytesBuilder,
        File,
        HttpClient,
        HttpClientRequest,
        HttpClientResponse,
        HttpHeaders,
        HttpRequest,
        HttpServer,
        InternetAddress,
        Process,
        SecurityContext,
        stderr,
        stdout;
import 'dart:convert' show utf8;

Future<void> shellCommand(String command) async {
  print('Executing command $command');
  final Process process = await Process.start('sh', ['-c', command]);
  stdout.addStream(process.stdout);
  stderr.addStream(process.stderr);
  final int exitCode = await process.exitCode;
  if (exitCode != 0) {
    throw new Exception('Process exited with status $exitCode');
  }
}

void main() async {
  // Last year's certificates:
  await shellCommand(
      'openssl req -newkey rsa:2048 -nodes -keyout ca2018.privatekey.pem -subj "/OU=CA" -days 731 -x509 -out ca2018.certificate.pem');
  // This year's certificates:
  await shellCommand(
      'openssl req -newkey rsa:2048 -nodes -keyout ca2019.privatekey.pem -subj "/OU=CA" -days 731 -x509 -out ca2019.certificate.pem');
  await shellCommand(
      'openssl req -newkey rsa:2048 -passout pass:password -keyout privatekey.pem -subj "/CN=localhost" -days 731 -sha256 -new -out csr2019.pem');
  await shellCommand(
      'openssl x509 -req -in csr2019.pem -CA ca2019.certificate.pem -CAkey ca2019.privatekey.pem -set_serial 1 -days 730 -sha256 -out certificate2019.pem');
  await shellCommand(
      'cat certificate2019.pem ca2019.certificate.pem > certificate2019.chain.pem');

  final SecurityContext serverSecurityContext = new SecurityContext();
  serverSecurityContext.useCertificateChainBytes(
      await new File('certificate2019.chain.pem').readAsBytes());
  serverSecurityContext.usePrivateKey('privatekey.pem', password: 'password');
  final HttpServer httpServer = await HttpServer.bindSecure(
      InternetAddress.loopbackIPv4, 0, serverSecurityContext);
  httpServer.listen((HttpRequest request) {
    request.response.write('body1');
    request.response.close();
  });
  print('Server listening at https://localhost:${httpServer.port}/');

  print('Making request.');
  final SecurityContext clientSecurityContext =
      new SecurityContext(withTrustedRoots: false);
  clientSecurityContext.setTrustedCertificatesBytes(
      await new File('ca2018.certificate.pem').readAsBytes());
  clientSecurityContext.setTrustedCertificatesBytes(
      await new File('ca2019.certificate.pem').readAsBytes());
  final HttpClient httpClient = new HttpClient(context: clientSecurityContext);
  final HttpClientRequest request = await httpClient.getUrl(Uri(
      scheme: 'https', host: 'localhost', port: httpServer.port, path: '/'));
  final HttpClientResponse response = await request.close();
  final List<int> bytes = await response.fold(new BytesBuilder(),
      (BytesBuilder bytesBuilder, List<int> bytes) {
    bytesBuilder.add(bytes);
    return bytesBuilder;
  }).then((BytesBuilder bytesBuilder) => bytesBuilder.takeBytes());
  final String contenType =
      response.headers.value(HttpHeaders.contentTypeHeader) ?? '';
  print('${response.statusCode} ${response.reasonPhrase} '
      'content-type="$contenType" body="${utf8.decode(bytes)}"');

  httpServer.close(force: true);
}

YudiH commented 5 years ago

Hi, relatively new to Flutter here (and programming in general). Only familiar with the more basic stuffs but I've now encountered the need to use a CertificatePinner such as this in flutter: https://square.github.io/okhttp/3.x/okhttp/okhttp3/CertificatePinner.html (I've successfully implemented this in my previous kotlin/java project in android studio)

All I have is the public key in the form of a string like shown below, nothing else: "sha256/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="

How do I go about achieving this? Sorry if there's already an answer because I don't seem to fully understand the finer details of what's been written above in order to achieve what I could in java with these few lines of code:

 String hostname = "publicobject.com";
 CertificatePinner certificatePinner = new CertificatePinner.Builder()
     .add(hostname, "sha256/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
     .build();

Thanks in advance

nioncode commented 4 years ago

Ideally, it should be possible to get the hash of the SubjectPublicKeyInfo string as described in https://www.imperialviolet.org/2011/05/04/pinning.html to avoid possible reinterpretation attacks when the raw public key is used on a different curve e.g.

This most likely is already possible by manually parsing the provided DER/PEM representation in the badCertificateCallback and then calculating the hash, but this is tedious and error prone. If dart could provide a subjectPublicKeyInfoSha256 getter, this would be ideal.

sahil-oyo commented 4 years ago

Ideally, it should be possible to get the hash of the SubjectPublicKeyInfo string as described in https://www.imperialviolet.org/2011/05/04/pinning.html to avoid possible reinterpretation attacks when the raw public key is used on a different curve e.g.

This most likely is already possible by manually parsing the provided DER/PEM representation in the badCertificateCallback and then calculating the hash, but this is tedious and error prone. If dart could provide a subjectPublicKeyInfoSha256 getter, this would be ideal.

@nioncode, badCertificateCallback returns X509Certificate object which contains der and pem property. However, the certificate does not contain the whole chain. I can only see the root CA and only contains sha256 key for root CA.

Ideally it should contain the whole keychain with leaf, intermediate and root certificates.

sahil-oyo commented 4 years ago

@mleonhard, I agree with you. Having key rotation enabled is the way to go. It is the recommended way and should be used in such scenarios.

The other solution where we let HttpClient invoke badCertificateCallback for all calls is also unusable. Ideally we should pin with leaf CA server key or an intermediate CA key but badCertificateCallback only gives us the root CA. This binds us to only use root CA public key for pinning. This is not very safe.

Other Platforms still allow some way of Public Key pinning. The only option we have in dart is to pin the actual certificate (or create a self signed certificate). Even though it is the best solution, there should be an alternative.

nioncode commented 4 years ago

@sahilpatel16 are you sure that the certificate returned in the badCertificateCallback is the root certificate? I only use this with a self-signed certificate at the moment, so I'm not 100% sure, but I'm pretty sure this should always give you the leaf certificate, otherwise this would be kind of pointless indeed.

EDIT: seems to be correct and the corresponding issue is https://github.com/dart-lang/sdk/issues/39425

TMSantos commented 4 years ago

Any update on this? It's impossible to pin our own certificate, the httpClient only returns CA certificate

wederchr commented 4 years ago

Certificate Pinning is possible using setTrustedCertificatesBytes. However, HPKP is not possible as you've already mentioned.

mleonhard commented 4 years ago

@yapcwed if you have figured out how to get certificate pinning to work in dartlang, please share details on how you accomplished it. When I tried to get it working, I was thwarted by these bugs:

35462 setTrustedCertificates is broken
39425 badCertificateCallback receives the server's CA certificate, not its leaf certificate. For dart clients to pin a certificate, the server must offer only the leaf certificate during TLS handshake. Web browsers trust root CA certificates. All public CAs use intermediate signing certificates. For web browsers to trust a signed leaf certificate, the intermediate CA's certificate must be sent by the server. This means that web browsers cannot connect to a server that offers only its leaf certificate. So to support both web browsers and cert-pinning dartlang clients, one must configure the server to listen on two IP addresses, one offering only the leaf cert and the other offering the cert chain. This is extra complexity & expense.

TMSantos commented 4 years ago

@mleonhard only way I found to retrieve the correct certificate was using HttpClientResponse object

HttpClientRequest request = await _httpClient.getUrl(Uri.parse(url));
HttpClientResponse response = await request.close();

Here you can access:

response.certificate.pem
response.certificate.subject

Etc and it has the correct certificate, instead of CA certificate only.

I somehow made this to be compatible with "http" by returning response like this:


    final streamedResponse = IOStreamedResponse(
        response.handleError((error) {
          final httpException = error as HttpException;
          throw ClientException(httpException.message, httpException.uri);
        }, test: (error) => error is HttpException),
        response.statusCode,
        contentLength:
            response.contentLength == -1 ? null : response.contentLength,
        isRedirect: response.isRedirect,
        persistentConnection: response.persistentConnection,
        reasonPhrase: response.reasonPhrase,
        inner: response);
    return Response.fromStream(streamedResponse);

I would prefer to do the pinning normally at badCertificateCallback, but until it is fixed, this was the only way I found to perform certificate pinning, at response level

mleonhard commented 4 years ago

@TMSantos Thanks for sharing your workaround. It checks the certificate after sending the request data. I want to verify the certificate before that happens. Is there a way to make the TLS connection first, check the server certificate, and then do an HTTP request over the TLS channel?

wederchr commented 4 years ago

@mleonhard Ah I forgot to mention that my use case is probably different. I have a flutter app that connects to a specific endpoint. To prevent MITM attacks I pin the gateway's certificate using setTrustedCertificatesBytes as follows.

  static http.Client getSecureClient(List<List<int>> certificates) {
    final context = SecurityContext(withTrustedRoots: false);
    certificates.forEach((cert) {
      context.setTrustedCertificatesBytes(cert);
    });
    return IOClient(HttpClient(context: context));
  }

TMSantos commented 4 years ago

@mleonhard that's the optimal scenario, is also what I want, but didn't find an workaround to do the check before sending the request data yet

rorystephenson commented 3 years ago

Any progress on this front? I've tried all the suggestions and if I'm understanding correctly public key pinning is not possible right now if you don't have the possibility to use your own CA, which I unfortunately don't. The two aforementioned bugs seem to render this impossible for now?

zdnet commented 2 years ago

Any progress on this issue? public key pinning is a very common way to aviod MIMT, suggest Dart could implement it ASAP officially. Thanks.

Ephenodrom commented 2 years ago

Just to hang in and connect these two issues : #47695

Access to full certificate chain would be great.

Ephenodrom commented 2 years ago

@TMSantos Have you thought about bundling OpenSSL within a flutter app and using s_client via ffi to fetch the certificates ? This should give access to the complete chain and ssl pinning would be possible !?

sebkoller commented 2 years ago

I just released a package that does public key pinning (SHA-256 of SPKI): https://pub.dev/packages/certificate_pinning_httpclient

But it only works on Android and iOS right now. It downloads the certificate chain once via a method channel and uses them with a SecurityContext.

vanlooverenkoen commented 1 year ago

We wanted to check the sha1 of the intermediate certificate & the common name of the leaf certificate. But this is also not possible

dart-lang / sdk

SSL Public key Pinning support for SecurityContext #35981

35462 setTrustedCertificates is broken