nyanzebra
commented
1 year ago Attempting to reproduce the situation described. Will update with findings after testing DPS with EST and x509.
Closed cookieofcode closed 1 year ago
nyanzebra
commented
1 year ago Attempting to reproduce the situation described. Will update with findings after testing DPS with EST and x509.
nyanzebra
commented
1 year ago From speaking internally, DPS only checks CN for provisioning, so it generally does not matter that we drop the other DN fields for the identity cert. This also appears to be a known issue that we are tracking internally to fix. In the meantime, if you don't require EST you can follow this: https://docs.microsoft.com/en-us/azure/iot-edge/how-to-provision-devices-at-scale-linux-x509?view=iotedge-2020-11&tabs=individual-enrollment%2Cubuntu.
cookieofcode
commented
1 year ago Thank you for your response @nyanzebra. The problem described above does not lie within DPS (or the device connection group), this works fine. It is a technical limitation of the current configuration of the EST Server (DigiCert IoT Device Manager), that the organization unit must be included in the subject of the CSR otherwise it will not be signed.
This limitation is because we're using X509 authentication to the EST Server (with the manufacturing/birth certificate) to obtain an operational certificate. Another field than the common name (in the subject DN) is then used by IoT Device Manager to match both certificates to the same device.
nyanzebra
commented
1 year ago @varunpuranik, should I mark this as bug? Also, do we have any alternative EST configurations that we can suggest?
jlian
commented
1 year ago It is a technical limitation of the current configuration of the EST Server (DigiCert IoT Device Manager), that the organization unit must be included in the subject of the CSR otherwise it will not be signed.
Is it possible to configure it to remove this constraint?
cookieofcode
commented
1 year ago It is a technical limitation of the current configuration of the EST Server (DigiCert IoT Device Manager), that the organization unit must be included in the subject of the CSR otherwise it will not be signed.
Is it possible to configure it to remove this constraint?
@jlian We're in contact with the provider and still waiting for a response. Currently, we are not able to remove this constraint and we're required to adjust the subject in the EST request. This results in a workaround which can not be used in production. Do you have an estimate about the timeline when a fix could be included in a LTS release?
jlian
commented
1 year ago We've triaged the bug and are looking into a fix, but not too sure that it'll make the next release (1.4.2) which needs to go out in time to take the latest .NET security patch and dependency updates. My guess would be that it'll take about a month for this fix to be released as part of 1.4.3. Do you need it sooner?
pebneter
commented
1 year ago Hi @jlian , I'm working on the same project as @cookieofcode . If we can have a fix in November, this would still be ok for us. We are not in production yet but want to advance with our prototype.
jlian
commented
1 year ago We've picked up the work and are currently forecasting late Oct to release this in 1.4.3.
jlian
commented
1 year ago Still targeting late Oct to release in 1.4.3.
MZDN
commented
1 year ago I am facing similar issue, when I trying to provision a device using EST server (libest from cisco) installed outside the machine where azure iot identity service (aziotctl 1.4.1 ) runs. When debugging the network traffic, I could see that the aziotctl sends the device cert request(CSR) but est-server was not able to validate the request. does it mean that the csr created by aziotctl is invalid? Please note that everything worked for me when i am having est-server and aziotctl(1.4.1) on the same vm.
jlian
commented
1 year ago Hey folks, the change is merged (thanks @onalante-msft).
Ideally, we could have you try it before we take it for the release:
@cookieofcode, @pebneter, and @MZDN do you think you could give it a try this week?
pebneter
commented
1 year ago H @jlian, thanks for the update! Unfortunately, @cookieofcode is out of office for a while and I'm not sure how to reproduce it.
MZDN
commented
1 year ago Hi @jlian, I gave a try, but unfortunately it does not fix. Please notice that the provided package version is 1.4.0. On the EST-server, I could see following in the log at each request from aziotctl: OSSL error: 139832112584448:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:../ssl/record/rec_layer_s3.c:1543:SSL alert number 42 This error means that: 42 | bad_certificate | A certificate was corrupt, contained signatures that did not verify correctly, etc.
jlian
commented
1 year ago @MZDN I think your issue is a bit different compared to the original, which is about the missing Subject DN fields such as the OU. AFAIK, the original issue affects IoT Edge using DigiCert IoT Manger EST provider.
Since you're trying this with libest, I believe the issue is different.
Could you please open a separate issue and fill out the bug report, so that we could properly diagnose? In it, please also include if you're following steps from https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-configure-est-server (which uses libest) and if there are deviations.
MZDN
commented
1 year ago Hi @jlian, I opened a new issue https://github.com/Azure/iot-identity-service/issues/483. I think it is more related to iot-identity-service project. Thanks
cookieofcode
commented
1 year ago Hey folks, the change is merged (thanks @onalante-msft).
Ideally, we could have you try it before we take it for the release:
1. Download the patched binary here https://github.com/Azure/iot-identity-service/actions/runs/3364093065 2. And then manually install following these steps https://azure.github.io/iot-identity-service/installation.html@cookieofcode, @pebneter, and @MZDN do you think you could give it a try this week?
Hi @jlian. Thank you for the patch.
I've tried the patch with the following versions:
ubuntu@ubuntu:/var/lib/aziot$ sudo dpkg -l | grep aziot
ii aziot-edge 1.4.0-1 amd64 Azure IoT Edge Module Runtime
ii aziot-identity-service 1.4.0~dev-1 amd64 Azure IoT Identity Service and related servicesAnd the following configuration:
[provisioning.attestation]
method = "x509"
registration_id = "GW-01-OP-001"
[provisioning.attestation.identity_cert]
method = "est"
subject = { CN = "GW-01-OP-001", OU = "GW-01" }
url = "https://<...>/.well-known/est/<...>"
identity_cert = "file:///<...>/identity_cert.pem"
identity_pk = "file:///<...>/identity_key.pem"Unfortunately, the intercepted call to the EST endpoint does not contain another field in the subject than the common name (the organizational unit is still missing).
In the mean time, we found a solution with the EST provider to omit the organizational unit in the certificate signing request. Regardless of this, it should be possible to configure an alternative subject in the certificate signing request.
jlian
commented
1 year ago @cookieofcode that's weird, I wonder what's the difference between your setup vs https://github.com/Azure/iot-identity-service/issues/455#issuecomment-1301845449?
@onalante-msft any ideas?
onalante-msft
commented
1 year ago @cookieofcode Have you re-run iotedge config apply? It is required to propagate the augmented attestation information to identityd.
github-actions[bot]
commented
1 year ago This issue is being marked as stale because it has been open for 30 days with no activity.
jlian
commented
1 year ago IoT Edge 1.4.8 is released and includes this fix (which is part of identity service 1.4.2)
Expected Behavior
The configured Distinguished Name (DN) in the "subject" configuration should be added to the Certificate Signing Request (CSR) using Enrollment over Secure Transport (EST) when using X509 (client) authentication with an identity certificate.
Current Behavior
Only the common name of the subject is included in the CSR of the identity certificate. Other fields such as "organization_unit" (OU) are ommited. The subject configuration is included for the EST bootstrap identity ("bootstrap_identity"), but not for the EST device identity ("identity").
Intercepted EST network traffic (aziot-certd) for "identity_cert":
Intercepted network traffic for "bootstrap_identity"
``` POST [https://<...>/.well-known/est/<...>/simpleenroll HTTP/1.1](https://<...>/.well-known/est/<...>/simpleenroll) content-type: application/pkcs10 content-transfer-encoding: base64 host: <...> content-length: 825 Decoded Payload (openssl req -in csr -inform der -noout -text): Certificate Request: Data: Version: 1 (0x0) Subject: CN = GW-01-OP-001, OU = GW-01 Subject Public Key Info: Public Key Algorithm: id-ecPublicKey Public-Key: (256 bit) pub: <...> ASN1 OID: prime256v1 NIST CURVE: P-256 Attributes: Requested Extensions: X509v3 Extended Key Usage: TLS Web Client Authentication Signature Algorithm: ecdsa-with-SHA256 <...> ```The EST server is configured to only accept CSR that contains an "organization_unit". This results in the error:
{"errors":[{"code":"invalid_input","message":"Please provide value for subject.organization_unit"}]}. This field is a constraint for an operational certificate by the PKI / EST server provider "DigiCert IoT Manger" (referenced in the documentation, see https://docs.microsoft.com/en-us/azure/iot-edge/tutorial-configure-est-server?view=iotedge-2020-11#est-server)Steps to Reproduce
Provide a detailed set of steps to reproduce the bug.
Context (Environment)
Configuration:
Output of
iotedge checkClick here
``` Configuration checks (aziot-identity-service) --------------------------------------------- √ keyd configuration is well-formed - OK √ certd configuration is well-formed - OK √ tpmd configuration is well-formed - OK √ identityd configuration is well-formed - OK √ daemon configurations up-to-date with config.toml - OK √ identityd config toml file specifies a valid hostname - OK √ aziot-identity-service package is up-to-date - OK √ host time is close to reference time - OK √ production readiness: identity certificates expiry - OK √ production readiness: EST identity and bootstrap certificates expiry - OK √ preloaded certificates are valid - OK √ keyd is running - OK √ certd is running - OK √ identityd is running - OK √ read all preloaded certificates from the Certificates Service - OK √ read all preloaded key pairs from the Keys Service - OK √ ensure all preloaded certificates match preloaded private keys with the same ID - OK Connectivity checks (aziot-identity-service) -------------------------------------------- ‼ host can connect to and perform TLS handshake with iothub AMQP port - Warning Could not retrieve iothub_hostname from provisioning file. Please specify the backing IoT Hub name using --iothub-hostname switch if you have that information. Since no hostname is provided, all hub connectivity tests will be skipped. ‼ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - Warning Could not retrieve iothub_hostname from provisioning file. Please specify the backing IoT Hub name using --iothub-hostname switch if you have that information. Since no hostname is provided, all hub connectivity tests will be skipped. ‼ host can connect to and perform TLS handshake with iothub MQTT port - Warning Could not retrieve iothub_hostname from provisioning file. Please specify the backing IoT Hub name using --iothub-hostname switch if you have that information. Since no hostname is provided, all hub connectivity tests will be skipped. √ host can connect to and perform TLS handshake with DPS endpoint - OK Configuration checks -------------------- √ aziot-edged configuration is well-formed - OK √ configuration up-to-date with config.toml - OK √ container engine is installed and functional - OK × configuration has correct URIs for daemon mgmt endpoint - Error SocketError - SocketErrorCode (TimedOut) : Operation timed out One or more errors occurred. (Got bad response: ) √ aziot-edge package is up-to-date - OK √ container time is close to host time - OK ‼ DNS server - Warning Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub. Please see https://aka.ms/iotedge-prod-checklist-dns for best practices. You can ignore this warning if you are setting DNS server per module in the Edge deployment. ‼ production readiness: logs policy - Warning Container engine is not configured to rotate module logs which may cause it run out of disk space. Please see https://aka.ms/iotedge-prod-checklist-logs for best practices. You can ignore this warning if you are setting log policy per module in the Edge deployment. ‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem. Data might be lost if the module is deleted or updated. Please see https://aka.ms/iotedge-storage-host for best practices. ‼ production readiness: Edge Hub's storage directory is persisted on the host filesystem - Warning The edgeHub module is not configured to persist its /tmp/edgeHub directory on the host filesystem. Data might be lost if the module is deleted or updated. Please see https://aka.ms/iotedge-storage-host for best practices. √ proxy settings are consistent in aziot-edged, aziot-identityd, moby daemon and config.toml - OK Connectivity checks ------------------- 24 check(s) succeeded. 7 check(s) raised warnings. Re-run with --verbose for more details. 1 check(s) raised errors. Re-run with --verbose for more details. 7 check(s) were skipped due to errors from other checks. Re-run with --verbose for more details. ```Device Information
Runtime Versions
iotedge version]: 1.3.0docker version]: 20.10.17+azure-1Logs
aziot-edged logs
``` Aug 09 01:07:57 ubuntu systemd[1]: Started Azure IoT Identity Service. Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [INFO] - Starting service... Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [INFO] - Version - 1.3.0 Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [INFO] - Provisioning starting. Reason: Startup Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 75 bytes Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::pool] pooling idle connection for ("http", keyd.sock) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 1 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is empty Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- GET /keypair/device-id?api-version=2021-05-01 {"host": "keyd.sock"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 357 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 1 headers Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is empty Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [INFO] - <-- GET /certificates/device-id?api-version=2020-09-01 {"host": "certd.sock"} Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [INFO] - !!! parameter "id" has an invalid value Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [INFO] - !!! caused by: not found Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [INFO] - --> 400 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 190 bytes Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 81 bytes Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (73 bytes) Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::pool] pooling idle connection for ("http", certd.sock) Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 369 bytes Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is empty Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::pool] pooling idle connection for ("http", keyd.sock) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- DELETE /keypair?api-version=2021-05-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "248"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 204 {} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 83 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (56 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /keypair?api-version=2021-05-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "56"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 174 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 357 bytes Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::pool] pooling idle connection for ("http", keyd.sock) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/algorithm?api-version=2021-05-01 {"content-length": "248", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 123 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/rsa-modulus?api-version=2021-05-01 {"content-length": "248", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 465 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/rsa-exponent?api-version=2021-05-01 {"content-length": "248", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 124 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/algorithm?api-version=2021-05-01 {"content-length": "248", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 123 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/rsa-modulus?api-version=2021-05-01 {"content-length": "248", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 465 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (248 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/rsa-exponent?api-version=2021-05-01 {"content-length": "248", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 124 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (355 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /encrypt?api-version=2021-05-01 {"content-length": "355", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 470 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 1080 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 1 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is empty Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- GET /keypair/est-id-device-id?api-version=2021-05-01 {"host": "keyd.sock"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 365 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (256 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/algorithm?api-version=2021-05-01 {"content-length": "256", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 123 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (955 bytes) Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [INFO] - <-- POST /certificates?api-version=2020-09-01 {"content-type": "application/json", "host": "certd.sock", "content-length": "955"} Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 82 bytes Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (256 bytes) Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::pool] pooling idle connection for ("http", keyd.sock) Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::connect::http] connecting to 192.168.117.1:8888 Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::connect::http] connecting to 192.168.117.1:8888 Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (256 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/rsa-modulus?api-version=2021-05-01 {"content-length": "256", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 465 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] parsed 2 headers Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (256 bytes) Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - <-- POST /parameters/rsa-exponent?api-version=2021-05-01 {"content-length": "256", "content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [INFO] - --> 200 {"content-type": "application/json"} Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::io] flushed 124 bytes Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-keyd[15506]: 2022-08-09T08:07:57Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::connect::http] connected to 192.168.117.1:8888 Aug 09 01:07:57 ubuntu aziot-certd[15512]: 2022-08-09T08:07:57Z [DBUG] - [hyper::client::connect::http] connected to 192.168.117.1:8888 Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::io] flushed 1043 bytes Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::io] flushed 122 bytes Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::io] parsed 12 headers Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::conn] incoming body is chunked encoding Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::decode] incoming chunked header: 0x64 (100 bytes) Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::client::pool] pooling idle connection for ("https", clientauth.demo.one.digicert.com) Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [ERR!] - !!! internal error Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [ERR!] - !!! caused by: could not create cert Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [ERR!] - !!! caused by: EST endpoint did not return successful response: 400 Bad Request b"{\"errors\":[{\"code\":\"invalid_input\",\"message\":\"Please provide value for subject.organization_unit\"}]}" Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [INFO] - --> 500 {"content-type": "application/json"} Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::io] flushed 155 bytes Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::io] parsed 3 headers Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::conn] incoming body is content-length (28 bytes) Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::conn] incoming body completed Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [DBUG] - [hyper::client::pool] pooling idle connection for ("http", certd.sock) Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: internal error Aug 09 01:07:58 ubuntu aziot-certd[15512]: 2022-08-09T08:07:58Z [DBUG] - [hyper::proto::h1::conn] read eof Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [ERR!] - service encountered an error Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [ERR!] - caused by: internal error Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [ERR!] - caused by: could not create certificate Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [ERR!] - caused by: internal error Aug 09 01:07:58 ubuntu aziot-identityd[15898]: 2022-08-09T08:07:58Z [ERR!] - 0:Additional Information
Please provide any additional information that may be helpful in understanding the issue.