Adding support for OpenSSL engines when using curl

[ghost]

All CLA requirements met.

[CIPop]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[CIPop]

@JonathanKnam this looks good to me! Thank you for the contribution!

I've started a test run to check for any regressions. If that passes, I need to enable existing OpenSSL ENGINE testing in the Azure IoT C-SDK repo and perform a manual run.

[JonathanKnam]

Even with fixes pointed out in the comments, the tests are still failing as no callback is made. Please investigate the IoT Hub Client and Provisioning Client tests. I have enabled HTTP tests in this branch: https://github.com/CIPop/azure-iot-sdk-c/tree/curl-openssl-engine

Ok, I'll have a look after I checked, how I need to setup the environment variables (especially the private key & certificates in the expected format), so that I can run the tests.

[CIPop]

Ok, I'll have a look after I checked, how I need to setup the environment variables (especially the private key & certificates in the expected format), so that I can run the tests.

Thank you @JonathanKnam !

The test suite for OpenSSL ENGINE is this one : https://github.com/Azure/azure-iot-sdk-c/blob/main/build/.vsts-ci.yml#L754 ('OpenSSL with pkcs11 engine and SoftHSM').

The docker image is also available here: https://hub.docker.com/r/aziotbld/openssl-pkcs11

I usually:

1. run the image interactively: docker run -it <image> /bin/bash
2. Clone the repo with submodules
3. Set the environment (ignore DPS for now)
   1. IOTHUB_CONNECTION_STRING - IoT Hub Owner connection string (tests will create new devices)
   2. IOTHUB_E2E_X509_CERT_BASE64 - your certificate PEM, base64 encoded (cat my.pem | base64 -encode)
   3. IOTHUB_E2E_X509_PRIVATE_KEY_BASE64 - your private key PEM (no password), base64 encoded
   4. IOTHUB_E2E_X509_THUMBPRINT - the certificate thumbprint (obtained with openssl x509 -in ./cert.pem -fingerprint -noout)
   5. IOTHUB_EVENTHUB_CONNECTION_STRING - the event hub connection string (obtained from IoT Hub built-in endpoints)
   6. IOTHUB_EVENTHUB_CONSUMER_GROUP - the event hub consumer group
4. Run the same scripts as the YML:
   1. sudo -E bash jenkins/linux_openssl_engine.sh
   2. cd cmake && sudo -E ../build_all/linux/run_tests.sh

[CIPop]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[CIPop]

A recent change to allow compilation on Ubuntu 22.04 requires adding new source files using OpenSSL ENGINE functions within the CMake exceptions script:

https://github.com/Azure/azure-c-shared-utility/commit/9e4322c9d88095ed276bf7e4b31b87b82b1c4d86#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20a

[CIPop]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[CIPop]

Thank you for your patience @JonathanKnam ! I fixed another memory leak for engine_id and found that the HTTP test was failing because of a test issue, not your code.

Everything appears to be working now end-to-end. I will be merging this PR then update the submodules within all Azure IoT C-SDK dependencies.

[CIPop]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).