Certificate and Private Key from a SmartCard

[Lethero]

So, I'm wanting to use your library to authenticate myself to a website over a SSL connection from a Windows machine. There is a certificate and a non-extracable private keys from a SmartCard that I can access using https://python-pkcs11.readthedocs.io/en/latest/applied.html. I'm able to use all the functions from the private key, I just can't export it to a file. Is there any support or advice you could give on going about doing that?

Getting the public, private, and certificates:

import OpenSSL
import pkcs11
from pkcs11 import Attribute, ObjectClass, KeyType

lib = pkcs11.lib(os.environ["PKCS11_MODULE"])
token = lib.get_token(token_label='smartcard')

with token.open(user_pin='a user pin'):
    pub_key = session.get_key(key_type=KetType.RSA, object_class=ObjectClass.PUBLIC_KEY)
    priv_key = session.get_key(key_type=KeyType.RSA, object_class=Objectlass.PRIVATE_KEY)

    certs = []
    for cert in session.get_objects({Attribute.CLASS: ObjectClass.CERTIFICATE}):
        certs.append(OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_ANS1, cert[Attribute.VALUE]))

[tomato42]

So this is not supported. That being said, there is nothing in the library that would make it impossible to implement.

What's needed is a class that either inherits from RSAKey: https://github.com/tomato42/tlslite-ng/blob/96f8287aede955b32f232361220b9935de3c0d1e/tlslite/utils/rsakey.py#L12-L24 like this pure-python one here: https://github.com/tomato42/tlslite-ng/blob/96f8287aede955b32f232361220b9935de3c0d1e/tlslite/utils/python_rsakey.py#L11-L14 or this one that uses m2crypto: https://github.com/tomato42/tlslite-ng/blob/96f8287aede955b32f232361220b9935de3c0d1e/tlslite/utils/openssl_rsakey.py#L32-L42

or a class that implements the same interface (for smartcards that may be actually a better approach as some of them do not allow raw private key operations for security reasons, but rather require the card itself to add the padding to the signed value) with regards to hasPrivateKey, __len__, hashAndSign, hashAndVerify, sign, verify, encrypt and decrypt

after this is implemented, then it should be enough to pass it as an argument to handshakeClientCert, like so: https://github.com/tomato42/tlslite-ng/blob/96f8287aede955b32f232361220b9935de3c0d1e/scripts/tls.py#L363-L364

since the certificate is something we need to send to the other side to be compliant with the protocol, it needs to be extractable, then it's just a question of converting that extracted object into x509 object, but this will be trivial compared with the above things

as far as requirements for it to be mergeable:

• the dependency on python-pkcs11 will need to be optional (the same way as dependency on m2crypto or pycrypto is)
• the test suite will likely need to use something like softhsm
• while it would also be nice for it to be able to handle ECDSA keys, I will not insist on that, but support for RSA-PSS will be necessary, we can't negotiate TLS 1.3 with RSA keys without it.
• since the dependency will be optional, the only requirement as far as supported python versions is that it needs to run on Python 3, if you want you can also make it work with older python 3 (as in <3.6) or with python 2 (either 2.7 or both 2.7 and 2.6)

[Lethero]

So, I'm running into a problem that the handshakes here aren't doing the mutual tls authentication. I know a certificate is required cause I'm prompted for a cert from my smart card in a browser.

[tomato42]

you need to pass the certificates as the first argument to handshakeClientCert, the certificates need to be wrapped in the X509CertChain object. Easiest way to do it is to get them in PEM format and then use the parsePemList() method

[Lethero]

import atexit
import os

import OpenSSL
import pkcs11
from pkcs11 import Attribute, Mechanism, ObjectClass
from tlslite import X509, X509CertChain, TLSConnection
from tlslite.utils.rsakey import RSAKey

def get_pin():
    """Method to return the user's pin,"""
    pass

LIB = pkcs11.lib(os.environ["PKCS11_LIB"])
TOKEN = LIB.get_token()
SESSION = TOKEN.open(user_pin=get_pin())

@atexit.register
def close_session():
    SESSION.close()

def get_cert():
    certs = SESSION.get_objects({Attribute.CLASS: ObjectClass.CERTIFICATE})
    certs = [OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_ASN1, c[Attribute.VALUE]) for c in certs]
    aliases = {}
    for cert in certs:
        x509_cert = X509()
        x509_cert.parse(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert).decode('utf-8'))
        subject = cert.get_subject()
        subject_cn = dict(subject.get_components())[b'CN'].decode()
        issuer = cert.get_issuer()
        issuer_cn = dict(issuer.get_components())[b'CN'].decode()
        aliases[issuer_cn] = (subject_cn, x509_cert)

    for index, alias in enumerate(aliases):
        print("{}) {}, Issued by {}".format(index+1, aliases[alias][0], alias))
    print("Q) Quit")

    selected_cert = None
    while selected_cert is None:
        try:
            index = input("Select your cert: ")
            if index.upper() == 'Q':
                print("Goodbye")
                return
            else:
                index = int(index) - 1
                if index >= 0 and index < len(aliases):
                    selected_cert = list(aliases.values())[index][1]
                else:
                    print("Invalid selection.")
        except Exception as e:
            traceback.print_exc()
            print("Invalid input, try again.", e)
            selected_cert = None

    return selected_cert

class Pkcs11RsaKey(RSAKey):
    """Work in progress."""
    def __init__(self, key):
        self.key = key

    def __len__(self):
        """Gets the length of the key in bits."""
        return self.key.key_length

    def sign(self, data, *args, **kwargs):
        print("sign", data, *args, **kwargs)
        result = self.key.sign(bytes(data), mechanism=Mechanism.SHA1_RSA_PKCS)
        print("sign", result)
        return result

    def verify(self, sigBytes, bytes, padding='pkcs1', hashAlg=None, saltLen=None):
        print("verify", sigBytes, bytes, padding, hashAlg, saltLen)
        result = super().verify(sigBytes, bytes, padding=padding, hashAlg=hashAlg, saltLen=saltLen)
        print("verify", result)
        return result

    def hashAndSign(self, bytes, rsaScheme='PKCS1', hAlg='sha1', sLen=0):
        print("hashAndSign", bytes, rsaScheme, hAlg, sLen)
        result = super().hashAndSign(bytes, bytes, rsaScheme, hAlg, sLen)
        print("hashAndSign", result)
        return result

    def hashAndVerify(self, sigBytes, bytes, rsaScheme='PKCS1', hAlg='sha1', sLen=0):
        print("hashAndVerify", sigBytes, bytes, rsaScheme, hAlg, sLen)
        result = super().hashAndVerify(sigBytes, bytes, rsaScheme=rsaScheme, hAlg=hAlg, sLen=sLen)
        print("hashAndVerify", result)
        return result

    def encrypt(self, bytes):
        print("encrypt", bytes)
        result = super().encrypt(bytes)
        print("encrypt", result)
        return result

    def decrypt(self, encBytes):
        print("decrypt", encBytes)
        result = self.key.decrypt(encBytes, mechanism=Mechanism.RSA_X_509)
        print("decrypt", result)
        return result

# Host and port is an address that requires SmartCard certs
addr = (host, port)
path = "/"

cert = get_cert()
cert_chain = X509CertChain([cert])

keys = list(SESSION.get_objects({Attribute.CLASS: ObjectClass.PRIVATE_KEY, Attribute.LABEL: key_label}))
key = Pkcs11RsaKey(keys[0] if len(keys) > 0 else None)

conn = None
try:
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect(addr)
    sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
    conn = TLSConnection(sock)
    settings = HandshakeSettings()
    settings.useExperimentalTackExtension = True
    conn.handshakeClientCert(cert_chain, key, settings=settings, serverName=addr[0])
    print("Session ID:", conn.session.sessionID)
    print("\tServer Name:", conn.session.serverName)
    print("\tMaster Secret:", conn.session.masterSecret)
    print("\tVersion:", conn.getVersionName())
    print("\tCipher:", conn.getCipherName(), conn.getCipherImplementation())
    print("\tCipher Suite:", conn.session.cipherSuite)
#     print("\tClient X.509 SHA1 Fingerprint:", conn.session.clientCertChain.getFingerprint())
    print("\tServer X.509 SHA1 Fingerprint", conn.session.serverCertChain.getFingerprint())
    if conn.session.tackExt:
        if conn.session.tackInHelloExt:
            emptyStr = "\n (via TLS Extension)"
        else:
            emptyStr = "\n (via TACK Certificate)"
        print("\tTACK", emptyStr)
        print(str(conn.session.tackExt))
    print("\tNext-Protocol Negotiated:", conn.next_proto)

    request = b'GET ' + path.encode() + b' HTTP/1.1\n'
    request += b'HOST: ' + addr[0].encode() + b'\n'
    request += b'\r\n'
    conn.write(request)
    data = conn.read()
except TLSLocalAlert as tla:
    if tla.description == AlertDescription.user_canceled:
        print(str(tla))
    else:
        raise
except TLSRemoteAlert as tra:
    if tra == AlertDescription.handshake_failure:
        print(str(tra))
    else:
        raise               
except Exception as e:
    traceback.print_exc()
    print(e)
finally:
    if conn is not None:
        conn.close()

Session ID: (Hidden)
    Server Name: (Hidden)
    Master Secret:  (Hidden)
    Version: TLS 1.2
    Cipher: aes256 pycrypto
    Cipher Suite: 61
    Server X.509 SHA1 Fingerprint (Hidden)
    Next-Protocol Negotiated: None
Traceback (most recent call last):
  File "C:\Users\WxCC_Admin\workspace\wxstream\ssl_test.py", line 221, in <module>
    data = conn.read()
  File "C:\Users\WxCC_Admin\Anaconda3\lib\site-packages\tlslite\tlsrecordlayer.py", line 188, in read
    for result in self.readAsync(max, min):
  File "C:\Users\WxCC_Admin\Anaconda3\lib\site-packages\tlslite\tlsrecordlayer.py", line 207, in readAsync
    for result in self._getMsg(ContentType.application_data):
  File "C:\Users\WxCC_Admin\Anaconda3\lib\site-packages\tlslite\tlsrecordlayer.py", line 722, in _getMsg
    raise TLSRemoteAlert(alert)
tlslite.errors.TLSRemoteAlert: handshake_failure
Closing session

From what I can follow, it gets through the exchange of keys, but never sends the cert over as the server doesn't request it. When I try to send the 'GET' request it appears to still be having a handshake type set for renegotiation and not application data. My Pkcs11RsayKey class is never called to authenticate either.

[tomato42]

From what I can follow, it gets through the exchange of keys, but never sends the cert over as the server doesn't request it.

without packet capture, I really can't help you with that – I can only suggest capturing a handshake from a browser and one from tlslite-ng and then comparing them, looking for differences: to guess the reason for the difference in server behaviour (SSLKEYLOGFILE may be useful)

maybe difference in hostname advertised in SNI?

does the server ask for handshake during the initial handshake, or with a renegotiation attempt (when connecting with a browser)?

When I try to send the 'GET' request it appears to still be having a handshake type set for renegotiation and not application data.

I'm quite sure it's not possible for write() to send handshake type messages... the very first thing the write() method does is pack the data into ApplicationData object: https://github.com/tomato42/tlslite-ng/blob/3ea78db9fe2be1f7dcc0fb9ab82201ef29657cb7/tlslite/tlsrecordlayer.py#L365-L383

side note:

settings.useExperimentalTackExtension = True

I'm pretty sure you don't want to use this...

[Lethero]

It doesn't ask for certificate during initial connection.

If it helps, I learned this server doesn't do the authentication, but handles authentication through another server. Do I need to do the handshake twice then?

[tomato42]

That does sound like renegotiation, unfortunately that is unsupported: #66

Would it be possible to make the server redirect users to different virtual host when asking for restricted resources? That would make it more compatible with TLS 1.3 at the same time (while there is protocol support for post-handshake authentication, the browsers do not enable it by default and some don't implement it at all).

[Lethero]

It appears to be doing post-handshake authentication and I noticed that work is ongoing for that. Just out of curiosity, what is the ETA on that?

[tomato42]

there is no ETA on #66, I'm focusing on TLS 1.3 exclusively now, after that I will probably be working on QUIC, so not in foreseeable future

[Lethero]

Alright, thank you for the quick responses.