Unable to Parse Private Key - Cannot Connect

[tafitzgerald]

• OS and version used: Linux Yocto 1.6.3
• Java runtime used: Zulu 1.8.0_202
• SDK version used: 1.19.1 - LTS_01_2020

Description of the issue:

I am trying to connect to IoT Central using the DeviceClient(conn string, protocol, publicCert, false, privateKey, false) constructor. I verified my CA with IoT Central and that succeeded. I created my private device key and signed my CSR with the CA.

When I try connecting, I get this error: https://github.com/Azure/azure-iot-sdk-java/blob/master/provisioning/security/x509-provider/src/main/java/com/microsoft/azure/sdk/iot/provisioning/security/hsm/SecurityProviderX509Cert.java#L223

My key should be an PrivateKeyInfo instance but is not being recognized.

Code sample exhibiting the issue:

Console log of the issue:

Here is my connection error: java.io.IOException: java.security.cert.CertificateException: java.io.IOException: Unable to parse private key, type unknown at com.microsoft.azure.sdk.iot.device.auth.IotHubX509SoftwareAuthenticationProvider.getSSLContext(IotHubX509SoftwareAuthenticationProvider.java:72) at com.microsoft.azure.sdk.iot.device.transport.mqtt.MqttIotHubConnection.open(MqttIotHubConnection.java:127) at com.microsoft.azure.sdk.iot.device.transport.IotHubTransport.openConnection(IotHubTransport.java:707) at com.microsoft.azure.sdk.iot.device.transport.IotHubTransport.open(IotHubTransport.java:273) at com.microsoft.azure.sdk.iot.device.DeviceIO.open(DeviceIO.java:158) at com.microsoft.azure.sdk.iot.device.InternalClient.open(InternalClient.java:133) at com.microsoft.azure.sdk.iot.device.DeviceClient.open(DeviceClient.java:334) at ... at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.eclipse.kura.web.server.OsgiRemoteServiceServlet.service(OsgiRemoteServiceServlet.java:41) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.equinox.http.servlet.internal.HttpServiceRuntimeImpl$LegacyServlet.service(HttpServiceRuntimeImpl.java:1223) at org.eclipse.equinox.http.servlet.internal.registration.EndpointRegistration.service(EndpointRegistration.java:148) at org.eclipse.equinox.http.servlet.internal.servlet.ResponseStateHandler.processRequest(ResponseStateHandler.java:62) at org.eclipse.equinox.http.servlet.internal.context.DispatchTargets.doDispatch(DispatchTargets.java:131) at org.eclipse.equinox.http.servlet.internal.servlet.ProxyServlet.service(ProxyServlet.java:74) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.equinox.http.jetty.internal.HttpServerManager$InternalHttpServiceServlet.service(HttpServerManager.java:284) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) Caused by: java.security.cert.CertificateException: java.io.IOException: Unable to parse private key, type unknown at com.microsoft.azure.sdk.iot.deps.auth.IotHubSSLContext.parsePrivateKey(IotHubSSLContext.java:308) at com.microsoft.azure.sdk.iot.deps.auth.IotHubSSLContext.generateSSLContextWithKeys(IotHubSSLContext.java:196) at com.microsoft.azure.sdk.iot.deps.auth.IotHubSSLContext.<init>(IotHubSSLContext.java:168) at com.microsoft.azure.sdk.iot.device.auth.IotHubX509SoftwareAuthenticationProvider.generateSSLContext(IotHubX509SoftwareAuthenticationProvider.java:102) at com.microsoft.azure.sdk.iot.device.auth.IotHubX509SoftwareAuthenticationProvider.getSSLContext(IotHubX509SoftwareAuthenticationProvider.java:62) ... 56 more Caused by: java.io.IOException: Unable to parse private key, type unknown at com.microsoft.azure.sdk.iot.deps.auth.IotHubSSLContext.getPrivateKey(IotHubSSLContext.java:376) at com.microsoft.azure.sdk.iot.deps.auth.IotHubSSLContext.parsePrivateKey(IotHubSSLContext.java:303) ... 60 more

Need Support?

• Have a feature request for SDKs? Please post it on User Voice to help us prioritize.
• Have a technical question? Ask on Stack Overflow with tag “azure-iot-hub”
• Need Support? Every customer with an active Azure subscription has access to support with guaranteed response time. Consider submitting a ticket and get assistance from Microsoft support team
• Found a bug? Please help us fix it by thoroughly documenting it and filing an issue on GitHub (C, Java, .NET, Node.js, Python).

[tafitzgerald]

For clarification:

I created my device private key and CSR with: openssl req -out sha1.csr -new -newkey rsa:2048 -nodes -keyout sha1.key

And then I signed my CSR with the CA with: openssl x509 -req -days 360 -in sha1.csr -CA <root-CA> -CAkey <root-key.pem> -CAcreateserial -out sha1.crt -sha256

[timtay-microsoft]

What format is your private key in? Is it something like the below?

-----BEGIN EC PRIVATE KEY-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
-----END EC PRIVATE KEY-----

[tafitzgerald]

No, my private key looks like below:

-----BEGIN PRIVATE KEY----- XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -----END PRIVATE KEY-----

I also tried generating using the bash script methods given from the azure-iot-sdk-c repo here, which generated my private key like:

-----BEGIN RSA PRIVATE KEY----- XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -----END RSA PRIVATE KEY-----

But this also fails for me with the same exception thrown.

[timtay-microsoft]

Have you tried using the generator located in this repo? You can find it here

[tafitzgerald]

I wasn't aware of that tool before, so thank you for showing that to me. I think the problem may have been with my decryption service. That could be why my private key was unrecognizable by the SDK.

Now I am having certificate errors being thrown. I get a "Not authorized to connect" MQTT exception when I try using the client cert:

2020-05-12T03:17:52,622 [qtp6552587-572] WARN c.m.a.s.i.d.t.m.Mqtt - Exception encountered while sending MQTT CONNECT packet Not authorized to connect (5) at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:28) at org.eclipse.paho.client.mqttv3.internal.ClientState.notifyReceivedAck(ClientState.java:1040) at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:151) at java.lang.Thread.run(Thread.java:748) . . . 2020-05-12T03:17:52,622 [qtp6552587-572] WARN c.m.a.s.i.d.t.m.Mqtt - Exception encountered while sending MQTT CONNECT packet Not authorized to connect (5) at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:28) at org.eclipse.paho.client.mqttv3.internal.ClientState.notifyReceivedAck(ClientState.java:1040) at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:151) at java.lang.Thread.run(Thread.java:748) 2020-05-12T03:17:52,686 [qtp6552587-572] INFO c.m.a.s.i.d.DeviceClient - Closing device client... 2020-05-12T03:17:52,693 [qtp6552587-572] INFO c.m.a.s.i.d.t.IotHubTransport - Client connection closed successfully 2020-05-12T03:17:52,695 [qtp6552587-572] INFO c.m.a.s.i.d.DeviceClient - Device client closed successfully 2020-05-12T03:17:52,696 [qtp6552587-572] INFO c.e.f.a.i.m.c.t.MqttDataTransport - Client closed 2020-05-12T03:17:52,701 [qtp6552587-572] WARN o.e.k.w.s.GwtNetworkServiceImpl - Error connecting org.eclipse.kura.web.server.GwtStatusServiceImpl.connectDataService(GwtStatusServiceImpl.java:135) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.eclipse.kura.web.server.OsgiRemoteServiceServlet.service(OsgiRemoteServiceServlet.java:41) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.equinox.http.servlet.internal.HttpServiceRuntimeImpl$LegacyServlet.service(HttpServiceRuntimeImpl.java:1223) at org.eclipse.equinox.http.servlet.internal.registration.EndpointRegistration.service(EndpointRegistration.java:148) at org.eclipse.equinox.http.servlet.internal.servlet.ResponseStateHandler.processRequest(ResponseStateHandler.java:62) at org.eclipse.equinox.http.servlet.internal.context.DispatchTargets.doDispatch(DispatchTargets.java:131) at org.eclipse.equinox.http.servlet.internal.servlet.ProxyServlet.service(ProxyServlet.java:74) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.equinox.http.jetty.internal.HttpServerManager$InternalHttpServiceServlet.service(HttpServerManager.java:284) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Could not open the connection at com.microsoft.azure.sdk.iot.device.DeviceIO.open(DeviceIO.java:162) at com.microsoft.azure.sdk.iot.device.InternalClient.open(InternalClient.java:133) at com.microsoft.azure.sdk.iot.device.DeviceClient.open(DeviceClient.java:334) ... 49 more Caused by: com.microsoft.azure.sdk.iot.device.transport.mqtt.exceptions.MqttUnauthorizedException: Unable to establish MQTT connection at com.microsoft.azure.sdk.iot.device.transport.mqtt.exceptions.PahoExceptionTranslator.convertToMqttException(PahoExceptionTranslator.java:63) at com.microsoft.azure.sdk.iot.device.transport.mqtt.Mqtt.connect(Mqtt.java:130) at com.microsoft.azure.sdk.iot.device.transport.mqtt.MqttMessaging.start(MqttMessaging.java:57) at com.microsoft.azure.sdk.iot.device.transport.mqtt.MqttIotHubConnection.open(MqttIotHubConnection.java:187) at com.microsoft.azure.sdk.iot.device.transport.IotHubTransport.openConnection(IotHubTransport.java:707) at com.microsoft.azure.sdk.iot.device.transport.IotHubTransport.open(IotHubTransport.java:273) at com.microsoft.azure.sdk.iot.device.DeviceIO.open(DeviceIO.java:158) ... 52 more Caused by: Not authorized to connect (5) at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:28) at org.eclipse.paho.client.mqttv3.internal.ClientState.notifyReceivedAck(ClientState.java:1040) at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:151) ... 1 more

And a different SSL Handshake exception when I try either the intermediate certificate or the root cert:

2020-05-12T03:18:45,178 [qtp6552587-560] WARN c.m.a.s.i.d.t.m.Mqtt - Exception encountered while sending MQTT CONNECT packet MqttException (0) - javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:38) at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:736) at java.lang.Thread.run(Thread.java:748) Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:994) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(SSLNetworkModule.java:149) at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:722) ... 1 more Caused by: java.io.EOFException: SSL peer shut down incorrectly at sun.security.ssl.InputRecord.read(InputRecord.java:505) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975) ... 6 more . . . Caused by: java.io.IOException: Could not open the connection at com.microsoft.azure.sdk.iot.device.DeviceIO.open(DeviceIO.java:162) at com.microsoft.azure.sdk.iot.device.InternalClient.open(InternalClient.java:133) at com.microsoft.azure.sdk.iot.device.DeviceClient.open(DeviceClient.java:334) at Caused by: com.microsoft.azure.sdk.iot.device.exceptions.ProtocolException: Unable to establish MQTT connection at com.microsoft.azure.sdk.iot.device.transport.mqtt.exceptions.PahoExceptionTranslator.convertToMqttException(PahoExceptionTranslator.java:47) at com.microsoft.azure.sdk.iot.device.transport.mqtt.Mqtt.connect(Mqtt.java:130) at com.microsoft.azure.sdk.iot.device.transport.mqtt.MqttMessaging.start(MqttMessaging.java:57) at com.microsoft.azure.sdk.iot.device.transport.mqtt.MqttIotHubConnection.open(MqttIotHubConnection.java:187) at com.microsoft.azure.sdk.iot.device.transport.IotHubTransport.openConnection(IotHubTransport.java:707) at com.microsoft.azure.sdk.iot.device.transport.IotHubTransport.open(IotHubTransport.java:273) at com.microsoft.azure.sdk.iot.device.DeviceIO.open(DeviceIO.java:158) ... 51 more Caused by: MqttException (0) - javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:38) at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:736) ... 1 more Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:994) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(SSLNetworkModule.java:149) at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:722) ... 1 more Caused by: java.io.EOFException: SSL peer shut down incorrectly at sun.security.ssl.InputRecord.read(InputRecord.java:505) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975) ... 6 more

[tafitzgerald]

@timtay-microsoft I haven't found any documentation on how to pass the certificate to the sdk using the verified root cert in Azure IoT Central. I figured that I would create a CSR locally and sign with the CA verified in Iot Central, but I can't seem to pass the right certificate to the sdk.

[tafitzgerald]

I used the provisioning generator and call DeviceClient(connStr, protocol, cert, false, privateKey, false) using the format described here but fail connection with a "Not authorized to connect (5)" exception.

[timtay-microsoft]

You should construct your device client using the constructor that takes a SecurityProvider, and you can use the same securityProvider instance that you use during the device provisioning portion of IoT Central. We don't have a great sample to show this, but if you share your project (minus the secrets), I could help you find what is wrong

[tafitzgerald]

I was not registering with the Security Provider before creating a connection. Thank you for pointing me in the right direction. If I continue to have issues, I'll post my connection code and maybe you will notice right away what I am doing wrong. Thanks for your help here!

[timtay-microsoft]

Sounds good. For the sake of keeping this issues channel clean, I'm going to close this issue. If you hit a new issue, feel free to open a new thread on this page

[az-iot-builder-01]

@tafitzgerald, @timtay-microsoft, thank you for your contribution to our open-sourced project! Please help us improve by filling out this 2-minute customer satisfaction survey