Adding certificate to ClientCertificates with fluent syntax

[Capsup]

Hi Darkl,

I'm attempting to connect to a router using wss, but am having issues getting the following exception once I attempt to open the channel:

{System.Net.WebSockets.WebSocketException: Unable to connect to the remote server ---> System.Net.WebSockets.WebSocketException: Unable to connect to the remote server ---> System.Net.Http.WinHttpException: A certificate is required to complete client authentication
   --- End of inner exception stack trace ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.WebSockets.WinHttpWebSocket.<ConnectAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.WebSockets.WebSocketHandle.<ConnectAsyncCore>d__21.MoveNext()
   --- End of inner exception stack trace ---
   at System.Net.WebSockets.WebSocketHandle.<ConnectAsyncCore>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.WebSockets.ClientWebSocket.<ConnectAsyncCore>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at WampSharp.WebSockets.WebSocketWrapperConnection`1.<Connect>d__10.MoveNext()}

using the following code:

            var factory = new WampChannelFactory();
            var channel = factory.ConnectToRealm( "wss://ipaddress/ws" )
                                 .WebSocketTransport( new Uri( SERVERADDRESS ) )
                                 .SetClientWebSocketOptions(x =>
                                 {
                                     x.ClientCertificates.Add( new X509Certificate2( "testslave.crt", "password" ) );
                                 } )
                                 .JsonSerialization( new JsonSerializer()
                                 {
                                     ContractResolver = new PrivateResolver()
                                 } )
                                 .Build();

loading the certificate seems to go fine, as can be seen from looking at the options CertificateCollection after adding it:

am I being stupid and missing something entirely here or is the fluent syntax not applying the given options correctly?

Any help appreciated.

[Capsup]

Actually, thinking about it a little more has left me wondering; what exactly is ClientCertificates supposed to do?

Is it an implementation of TLS Certification Authentication from the WAMP Protocol, such that you can authenticate with the router using a TLS certificate instead of CRA (which is what I was looking for)? If yes, where do you actually add the authid for authentication? If not, I'm unsure what it is supposed to do.

And why does wss not work without a client certificate. It's using HTTPS after all, which doesn't have a requirement for a client certificate. So is this issue actually on the router-side (I'm using Crossbar.io, setup to use TLS), but I'm just getting a weird exception message?

[darkl]

These are options of the ClientWebSocket class, see here.

The ClientWebSocket.Options property is set as expected by the SetClientWebSocketOptions extension method.

You need to make sure both client and router use the same security certificates.

Elad

[Capsup]

But that's what I find curious. Shouldn't wss work, even though the client doesn't present a certificate? In that case, shouldn't they negotiate an encryption key using the server's certificate and then continue communication encrypted with that key?

But yeah, I'll fiddle with it a bit more and see what I end up with.

[darkl]

I think this property is intended for self-signed certificates.

[darkl]

Closing due to inactivity. Please comment here with your findings.

[Capsup]

@darkl My bad, I've just been spending time on a lot of other things because no matter what I did, I couldn't get this to work. WSS doesn't seem to work with WampSharp together with the Crossbar router. I can't seem to figure out why WSS won't just negotiate an encryption key with the server and then move on. I am using the same self-signed certificate on both ends, but the ClientWebsocketClass seems to not want to even attempt an connection, without having that property set, when attempting to connect to a WSS endpoint.

I'll get back to you as soon as I dive deeper into it. It has to be possible somehow.

[Vito303]

For the self-signed certificate, this way works for us.

            IWampChannelFactory factory = new WampChannelFactory();
            IWampChannel channel = factory.ConnectToRealm(realm)
               .WebSocketTransport(uri)
               .SetSecurityOptions(o =>
               {
                   o.EnabledSslProtocols = SslProtocols.Tls | SslProtocols.Tls11 | SslProtocols.Tls12;
                   //o.Certificates.Add(new X509Certificate2(@"server_cert.pem"));
                   o.AllowNameMismatchCertificate = true;
                   o.AllowUnstrustedCertificate = true;
               })
               .JsonSerialization()
               .Authenticator(authenticator)
               .Build();

            //// This is also neccessary
            ServicePointManager.ServerCertificateValidationCallback = (s, crt, chain, policy) => true;

[StevenBonePgh]

@Vito303 - What is the reason you are using secure sockets? By removing all safeguards against talking to potentially malicious middlemen or endpoints, you are designing and implementing an insecure solution that makes the use of SSL/TLS merely window dressing. A better (but not perfect) approach if you do not have the ability to purchase certificates is to create a signing authority certificate that is secret and shared between client/server, trust it from both client and server side, and issue certificates from it that the server hosts, using subject alternative names in the certificate to include the endpoint DSN(s). This is an imperfect solution as you cannot merely invalidate a compromised certificate with the issuing agency, but it is leaps and bounds more 'secure' than what you propose. This is off-topic for this library, and I am far from an expert on this topic, but I cringe when I see an obviously insecure example posted. Check out BouncyCastle as it has the APIs needed to perform such tasks. Also see this blog post and others in the series by the same author for additional guidance. Also useful is this and this.

[Capsup]

@darkl

so I'm back at this in the newest version of .NET Core and ... it still doesn't work and I'm about to start pulling my hair out of sheer frustration. The error messages given by the framework are an utter disgrace and shouldn't even qualify for an actual error message:

"System.Net.Http.WinHttpException (0x80072F8F): A security error occurred"

I'm not even sure what to try anymore. Even turning off the security measures as @Vito303 suggested:

 var factory = new WampChannelFactory();
            var channel = factory.ConnectToRealm( Configuration[ "RouterRealm" ] )
                .WebSocketTransport( new Uri( Configuration[ "RouterURL" ] ) )
                .SetClientWebSocketOptions( config => 
                {
                    config.ClientCertificates.Add( new System.Security.Cryptography.X509Certificates.X509Certificate2( @"xxx.pfx", "yyy" ) );
                } )
                .JsonSerialization()
                .Authenticator( new WampTicketAuthenticator() )
                .Build();

            ServicePointManager.ServerCertificateValidationCallback += ( s, crt, chain, policy ) => true;
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
            channel.Open();

I'm still running into the same issue. I'm getting no connection messages on the Crossbar router, so I'm assuming the request isn't even being sent to the webserver, since the ServerCertificateValicationCallback is never called. I'm using a custom self-signed certificate to connect with. It's signed by an intermediate certificate, created from a custom root ca. The router is using an equivalent signed certificate. Disabling SSL on the router and changing to a ws:// URL works fine. I can't seem to find the NameMismatch options anymore on the new WebSocketOptions configuration object. I can't debug into the .NET Core pdb file to see what actually goes wrong on the framework level, this is literally all the debug information I can manage to get at. Suggestions on how to debug this on the deeper framework levels are very welcome. It would help a lot just seeing the flow fail in the debugger. What I have yet to try is to make a quick autobahn|JS client and see if that works.

@StevenBonePgh do you happen to know what happened to the SSL Security Options? I know you're saying it's a bad idea to disable the security validation, and I completely agree, but at this point I just want it to connect and then I can start figuring out how to actually enable them again.

Any help greatly appreciated.

[StevenBonePgh]

First, lets fix up your language. It is not a custom self-signed certificate if it is signed by an intermediate authority. If you 'made up' the signer's certificate now you need to make sure that certificate is trusted as a signer of other certificates (generally at the OS level). I suppose you can say the certificate is signed by an untrusted authority. Providing a certificate when attempting to connect only should work with a truly self-signed certificate.

I gather your problem is most likely because you need to trust the certificate authority that signed the certificate and all intermediate signers up to your certificate. What platform and runtime versions is your client code running? Could you trust the signing intermediate certificate outside of code as part of configuration?

I'm not in a good place with TLS 1.2 and certificates right now, as you can tell from my Stack Overflow question of a few weeks ago. As described in my question, on Xamarin Android I can trust the signer at the OS level, but not in code at runtime - I'm certain this is due to the implementation of TLS 1.2 in Mono/Android. I am not seeing ServerCertificateValidationCallback being called either. While you are looking at SO, if you think my question is a good question, an upvote wouldn't hurt visibility and based on your platform some of the links in the question may be helpful as well. Good luck.

[darkl]

@esso23, can you please give any advice to @Capsup?

Thanks

[Capsup]

@StevenBonePgh

right, so signed by an intermediate authority that was signed by a certificate authority that were both generated by me for this specific purpose. I use .NET Core 2.1.2 right now to execute the client code.

Does that mean it would behave differently if I were to use a self-signed certificate instead of one signed by my own root CA? I haven't tried just a self-signed cert, but will definitely do that tomorrow.

According to your SO post, adding a CA signed certificate to the ClientCertificate shouldn't work. Why is that exactly? And if it doesn't, would it then be the correct understanding that I wouldn't actually need to supply a certificate on the clientside as long as the serverside uses a certificate signed by a trusted CA on the clientside?

Trusting the certificates as part of configuration is definitely possible in my particular use case. I'll look more into that, thanks for the idea.

[StevenBonePgh]

.NET Core on Windows? I'm shocked ServerCertificateValidationCallback is not being called (I haven't thought to try it myself, having the issue on Xamarin/Android which is Mono based). NetCore on Windows is a different backend stack.

The easiest way on Windows that I know of to determine trust of a certificate is to open the .cer file for a certificate in the Explorer UI - it will tell you details about validating the chain. You would need to trust the root CA and the intermediate CA then all certs issued by the root or intermediate will be accepted. When talking https/wss there is also a check on the cert name and SANs in the cert against the name listed in the URI. The github repo on my account has some code that looks at your hostname and ip addresses to issue a cert from an authority with a comprehensive SAN list.

I did not expect providing the CA Cert would accomplish anything, as my expectation is the cert is most likely checked for equivalence with the server, not as a cert-store replacement. That said, I would not fail to test adding the CA, Intermediate, and Client cert (yes, all three) to the collection and see what happens. In my case nothing, but your case is a different technology stack.

[Capsup]

I was indeed surprised as well, but it leads to me believe that my request isn't even being sent to the server, so it doesn't ever get around to validating the server's certificate.

This might be me just being stupid, but what exactly do I put into the cert name? Does the client get the SAN of "localhost" and the server the SAN of its' IP? That's how I've done it right now at least.

[StevenBonePgh]

Normally a certificate is issued for a single domain name, like 'example.com'. Such a certificate would protect https://example.com but not https://www.example.com. For supporting multiple hostnames, your certificate should contain a SAN (subject alternative name) entry (wildcards are allowed up to one level deep, so you could have *.example.com which would protect www.example.com, mail.example.com, etc.). Wikipedia has a surprisingly terrible entry for it, but it does link to wildcard certificate which is decent.

I would guess the request has made it to the server, otherwise you would not get the exception. I also just noticed that I did not answer your 2 back question:

And if it doesn't, would it then be the correct understanding that I wouldn't actually need to supply a certificate on the clientside as long as the serverside uses a certificate signed by a trusted CA on the clientside?

That is how it should work, and what I've done at multiple physical locations in a previous employ. I have a small server with a few clients that connect to it, all the clients are under my control. It was not exactly an inventory management solution, but this is the best example I can say it was closest to. In any case, imagine you have one customer with several locations, each location having a server, and therefore needing a certificate, because, hey, everyone expects modern cryptography with mobile applications. It does not make sense to pay for a certificate for each location, given the clients are completely controlled. Generating a CA per client, and having each location able to 'issue' a cert for that location from that CA means I only need to configure the all mobile clients to trust one CA and they can be moved freely from one location to another by the client with minimal configuration. In this rather antiquated environment I was able to achieve this in a manner that was local to the application and did not involve an OS level trust of the CA.

This is not my use case these days, but it is similar enough in that I would like to validate and protect a connection between an app and a local server, but the trust of the connection should be only local to the app. I do not wish to force a device, potentially used for other purposes, to 'trust' a CA that does not have the ability to revoke a certificate in the event of compromise, nor would I have access, via the app, to remove the CA. It is not really all that important for me (test scenario is my only use case, real deployments would use real certificates), but disturbing that there is no app-level solution on Xamarin/Android.

[Capsup]

Okay, so, after Steven's thorough explanation of how to correctly use TLS, it ended up actually working just manually adding the intermediate and ca to my windows' certificate store. Thanks a lot for your help Steven, now I just gotta get onto the actual issue of how to authenticate using a SSL certificate. (By the way, the ServerCertificateValidationCallback still isn't being called)

@darkl Elad, what's the correct way in WampSharp to authenticate using an SSL certificate? Adding the certificate to the ClientCertificate's Options object doesn't seem to pass it along as Crossbar router would expect. I get the error "client did not send a TLS client certificate", since it's expecting the certificate to be in the transport object of session_details, under the "client_cert" property, which appears to be null.

Is this the property that the Crossbar router should have deserialised the ClientCertificate property into, but doesn't?

I've also tried with a IWampClientAuthenticator that implements an AuthMethod of "TLS", but the Authenticator's Authenticate method is never actually called, leading me to believe that there's somewhere else where I need to present the certificate. Where is that, exactly?

[StevenBonePgh]

Glad to hear you are now connecting with TLS. I use CRA to authenticate, so I never looked at the other options available. I can tell you that some software that use certificate files are not compatible with the .cer binary format and require a .pem format. The GitHub repo above has a utility method to convert a loaded certificate to .pem format if you need it (I think using BouncyCastle). More on certificate formats on Server Fault.

[darkl]

@Capsup, I'm not very familiar with certificates, but WampSharp just uses the ClientWebSocket api, and therefore I guess you might want to try figuring out first how to set up client certificates for a ClientWebSocket instance. I would guess that is done by setting the ClientWebSocket.Options.ClientCertificates property, which can be done via WampSharp using the SetSecurityOptions extension method, but maybe something else is needed to be done.

Elad

[darkl]

I did a bit googling and found these resources: regarding your new exception and regarding your original exception (see threads, maybe you can also use Wireshark). Regarding crossbario, did you try to follow the instructions on their website? Did you try any of the detailed examples, such as this one?

Elad

[Capsup]

@darkl I am indeed using the crossbar examples as inspiration. The problem might be on Crossbar's side, but I don't know yet. I've tried debugging it a little with the Crossbar source code, but Python isn't really my strong side. I can't really find the certificate anywhere in the information that is given as part of the transport object there, so it seems like it isn't actually using the information that the .NET Core websocket stack is presenting as part of the ClientCertificate property?

I've already created certificates, properly configured the router and checked for .NET Core v2.1.

The Crossbar TLS authentication mechanic looks for the client certificate in the transport_details.transport.client_cert property. That one doesn't exist, even though I present my certificate in the ClientCertificate property in .NET. So the issues that I'm running into seems to be on the intersection between the .NET WebSocket implementation and the one that the Crossbar router uses.

I'll keep digging into it, thanks for your help so far.

[Capsup]

Aha! I seem to have finally figured it out.

I found this issue where it was mentioned that .NET Core just silently ignores certificates that it deems "insecure" or otherwise filters out. From there, it was mentioned that it is possible to read the tracing logs directly from the .NET Core framework using perfview and doing that, I found this log message.

So apparently something was filtering out my certificate, because just above it, it clearly says that it selects my certificate for security.

So looking at the open-source code of the .NET Core framework (Thanks a lot Microsoft!) for the SecureChannel class as mentioned in the log, on lines 449-508, it checks if the given client certificates are issued by one of the eligible issuers, given by the server.

I looked into the config of Crossbar and saw that I hadn't actually added the ca or the intermediate certificate to the "ca_certificates" property of the tls endpoint, thus the .NET Core SecureChannel was ignoring my client certificate, because it wasn't actually issued by one of the given issuers eligible from the server (because there weren't any..)!

I guess that's what I get from not just using the actual example config given or even just testing it manually with one of the given javascript examples, because they would have failed too, probably.

On the plus side, I've finally figured out how to debug directly with the .NET Core source files and even getting runtime logs, which will definitely come in handy for future debugging.

[darkl]

Great to hear that you've figured it out!