Any support for PKCS#11 compatible Hardware Security Module (HSM)?

[theinhtut]

Hello. Does paho.mqtt support a PKCS#11 compatible Hardware Security Module (HSM)?

My device private key is currently stored in HSM or softHSM2. According to the documentation, keyfile is a string pointing to the PEM encoded private key.

Since device's private key is stored inside HSM, its private key cannot be retrieved. Is there any workaround to support HSM instead of using PEM encoded file from local storage? Thanks in-advance☺

...
# Using PEM encoded file
client.tls_set(
    ca_certs=PATH_TO_ROOT_CERT,
    certfile=CERT_FILE,
    keyfile="./device-private-key.pem",
    cert_reqs=ssl.CERT_REQUIRED,
    tls_version=ssl.PROTOCOL_TLSv1_2,
    ciphers=None,
)

...
# Using HSM or softHSM2 with PKCS#11
client.tls_set(
    ca_certs=PATH_TO_ROOT_CERT,
    certfile=CERT_FILE,
    keyfile="pkcs11:object=deviceprivkey;type=private;pin-value=mypin",
    cert_reqs=ssl.CERT_REQUIRED,
    tls_version=ssl.PROTOCOL_TLSv1_2,
    ciphers=None,
)

References

AWS IoT MQTT with PKCS#11 support

[MattBrittan]

You can pass a ssl.SSLContext directly to client.tls_set_context; I think this may be used to do what you require.

HSM's are not something this library supports directly, and I don't see it being added unless there is more demand and a simple example showing the approach.

I'll leave this open for now; but unless you are able to provide more info (or others step in) it will probably be closed in a month or so (apologies that it's taken so long to provide a reply!).

[IniterWorker]

I've implemented a way to support PKCS#11 in the paho-mqtt Python library, building on my experience adapting azure-iot-sdk-python, which also relies on paho-mqtt. In my modifications, I've focused on enabling compatibility with both SSL.connection from PyOpenSSL and ssl.SSLContext. However, I'm currently deliberating whether to make PyOpenSSL a mandatory dependency.

As I'm not deeply entrenched in the Python ecosystem, I'd appreciate guidance from @MattBrittan or @jamesmyatt and/or any other party, on best practices for handling this new feature inside this library. I'm prepared to contribute and would like to ensure my changes align with the project's goals. Thank you in advance for your assistance

I will share a bit of code to help our collaboration:

    def tls_set_context(
        self,
        context: ssl.SSLContext | SSL.Context | None = None,
    ) -> None:
        """Configure network encryption and authentication context. Enables SSL/TLS support.

        :param context: an ssl.SSLContext or OpenSSL.SSL.Context object. By default, this is given by
            ``ssl.create_default_context()`` if available.

        Must be called before `connect()`, `connect_async()` or `connect_srv()`.
        """
        if self._ssl_context is not None:
            raise ValueError('SSL/TLS has already been configured.')

        if context is None:
            context = ssl.create_default_context()

        self._ssl = True
        self._ssl_context = context

        # Ensure _tls_insecure is consistent with check_hostname attribute for ssl.SSLContext
        if isinstance(context, ssl.SSLContext) and hasattr(context, 'check_hostname'):
            self._tls_insecure = not context.check_hostname
        elif isinstance(context, SSL.Context):
            # PyOpenSSL Context does not have check_hostname attribute
            # Set _tls_insecure based on custom logic if necessary
            self._tls_insecure = False  # Assuming default to False for PyOpenSSL

[MattBrittan]

Sorry - I'm not really qualified to comment (just trying to help by closing some issues). @PierreF has been the most active committer recently so may be able to assist.