Closed tymtam2 closed 2 years ago
Tagging subscribers to this area: @dotnet/ncl, @vcsjones See info in area-owners.md if you want to be subscribed.
Author: | tymtam2 |
---|---|
Assignees: | - |
Labels: | `area-System.Net.Security`, `untriaged` |
Milestone: | - |
I am afraid I can't reproduce this behavior:
using System.Net.Security;
using System.Security.Authentication;
using System.Security.Cryptography.X509Certificates;
var handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.SslProtocols = SslProtocols.Tls12;
handler.ServerCertificateCustomValidationCallback = (HttpRequestMessage message, X509Certificate2? certificate, X509Chain? chain, SslPolicyErrors sslPolicyErrors)
=>
{
System.Console.WriteLine("Got here");
return true;
};
handler.ClientCertificates.Add(new X509Certificate2(fileName: @"C:\source\DotnetSandbox\contoso.com.pfx", password: "testcertificate"));
var client = new HttpClient(handler);
var result = await client.GetAsync("https://self-signed.badssl.com/").ConfigureAwait(false); // no exception here
the client certificate should not even be considered unless server explicitly asks for client certificate (it is not sent by default by clients).
Can you perhaps create a self-contained repro (a small project) which we can download and run locally to reproduce the issue?
This issue has been marked needs-author-action
and may be missing some important information.
can you do packet capture @tymtam2? ssl3_read_bytes:tlsv1 alert unknown ca.
smells like server no liking you client certificate. e.g. this is not client problem.
It can also be because the server needs intermediate certificates from the pfx. You would see that as well if you compare it with curl
.
@wfurt > smells like server no liking you client certificate. e.g. this is not client problem.
I'd say that in this case it would not work with the same file from Rust, Postman, curl and with openssl s_client?
Different errors in linux and Windows strongly suggest for me it's a client issue, not the server's.
I'll keep investigating and report back.
looks what is in that pfx. You can always create standalone repo...
Thanks for trying to reproduce this. I think https://self-signed.badssl.com is not a good test case because this server doesn't require a client cert so the handshake is different. I'm not that comfortable in this area but if the client cert is not sent by default - which you point out - then using https://self-signed.badssl.com won't use the same path.
I wonder if you have a comment on why would the same pfx file work with curl, Rust, Postman and also with openssl s_client? With these I can connect successfully using the same file [the same file on the hard drive to eliminate any file issues)
For "raw" openssl the successful connection it achieved using:
openssl s_client -connect serverX.com.au:443 -cert client.crt -key priv_key_client.pem -CAfile client_chain.crt
Does it work for you @tymtam2 if you skip client_chain.crt
? You certainly don't do that pat with your c# snippet.
can you do packet capture @tymtam2?
I'll look into that as well, but is there something I could do to get more logs from httpClient(Handler) to diagnose this?
https://github.com/dotnet/runtime/issues/20671 says that
"starting with .NET Core 2.1 Preview 2, the default HTTP implementation uses the new SocketsHttpHandler (not WinHTTP)"
and this feels me with hope of being able of getting more logs (at least on Windows) :).
Yes, you can get more logs. But typically they don't cover details of the handshake. I feel packet captures and test without the chain would be most valuable for triage.
test without the chain
What is "test without the chain"?
From the above
openssl s_client -connect serverX.com.au:443 -cert client.crt -key priv_key_client.pem -CAfile client_chain.crt
change it to
openssl s_client -connect serverX.com.au:443 -cert client.crt -key priv_key_client.pem
Does it work for you @tymtam2 if you skip client_chain.crt? You certainly don't do that pat with your c# snippet.
It's part of the snippet via:
// File is output of "openssl pkcs12 -export -out withchain.pfx -inkey priv_key_client.pem -in client.crt -certfile chain.crt
With openssl s_client -connect serverX.com.au:443 -cert client.crt -key priv_key_client.pem
the it results in:
Verify return code: 20 (unable to get local issuer certificate)
Does it work for you @tymtam2 if you skip client_chain.crt? You certainly don't do that pat with your c# snippet.
It's part of the snippet via: // File is output of "openssl pkcs12 -export -out withchain.pfx -inkey priv_key_client.pem -in client.crt -certfile chain.crt
I wonder if you're saying that the chain cert needs to be supplied in a different way with HttpClientHandler and one pfx is not enough (I know I keep banging about it but this one pfx - with 3 certs [client, and two in the chain] and the private key - is enough when using Rust, postman and curl).
yes. But I wanted some evidence before making the claim. Server may be ok without the intermediates so seeing the exchange for both cases is useful. So as test with other clients where the software does not have access to the intermediates.
Basically you would need to do
X509Certificate2Collection certs = new X509Certificate2Collection("withchain.pfx");
and add that to User's "Ca" store e.g. places where intermediates are stored. There is currently no good way to pass them in directly.
After adding the crt with the CA cert chain it works:
ServerCertificateCustomValidationCallback
is calledAfter this spectacular success, I also checked if using a client.pfx that doesn't have the ca chain works, and it does.
A working snippet would be something like this:
var handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.SslProtocols = SslProtocols.Tls12;
handler.ServerCertificateCustomValidationCallback += (HttpRequestMessage message, X509Certificate2? certificate, X509Chain? chain, SslPolicyErrors sslPolicyErrors)
=>
{
System.Console.WriteLine("Checking server cert");;
return true;
};
// Change 1: (Optional) Not using ca_chain.crt so this may be either of:
// openssl pkcs12 -export -out client.pfx -inkey priv_key_client.pem -in client.crt -certfile ca_chain.crt
// openssl pkcs12 -export -out client.pfx -inkey priv_key_client.pem -in client.crt
var clientCertFileName = "client.pfx"; //pfx
var clientCertPKandChain = new X509Certificate2(fileName: folder + clientCertFileName, password: passphrase);
handler.ClientCertificates.Add(clientCertPKandChain);
// Change 2: register ca chain into the store:
var caChainCertFileName = "ca_chain.crt"; //crt
var caChainCert = new X509Certificate2(fileName: folder + caChainCertFileName);
var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadWrite);
store.Add(caChainCert);
var client = new HttpClient(handler);
var result = await client.GetAsync("https://serverX.com.au:443/dcap").ConfigureAwait(false); // Success
Observations:
ServerCertificateCustomValidationCallback
seem not that powerful ;)X509Store
s (current user's or local machine's) be writeable from an Azure Function? (What if there many running in parallel? How to manage adding the cert?) (I will of course check this, but a comment would be appreciated)This will eventually improve with https://github.com/dotnet/runtime/issues/71194 but I generally agree with the observation. I think it would still be interesting to see the packet captures with original issue but I understand that you may not want to since you have working solution.
Linux really does not have concept of certificate stores (unlike macOS and Windows). All the user stores are emulated on top of files in home directory - but I don't know if that is available from AF. Adding in parallel should be safe AFAIK. The certificates are indexed by hash and as far as I know adding existing certificate does not create another copy.
BTW You can also probably fix it with adding the intermediates to the server ... if you have control over it. RFC says client should send the intermediates but the TLS does not really depend on it.
I somewhat have control (via email to another organisation) and this may be the way to address this.
In my somewhat rudimentary understanding of TLS (1.2) the intermediates should not play a role. However, in work not related to what's happening here we've discovered that we need to supply the intermediate certs when connecting to Azure Device Provisioning Service even though the intermediate certs are uploaded to the DPS via a separate channel. We've just learned to live with this.
Apropos packet capture, what are you after? I should be able to set it up after the weekend. I have control only over the client so the data would be only from the client's perspective. Would this be still of interest?
Yes, client side will be sufficient. My suspicion is that the server is failing validate client's certificate without the intermediates and therefore aborting the session. We should see that when looking at the TLS exchange.
This log is from Windows and results in System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> System.ComponentModel.Win32Exception (0x80090325): The certificate chain was issued by an authority that is not trusted.
"No.","Time","Source","Destination","Protocol","Length","Info"
"22149","1090.883660","192.168.1.23","se.rv.er","TCP","66","56474 > 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 WS=256 SACK_PERM=1"
"22150","1090.902054","se.rv.er","192.168.1.23","TCP","66","443 > 56474 [SYN, ACK] Seq=0 Ack=1 Win=64240 Len=0 MSS=1460 SACK_PERM=1 WS=128"
"22151","1090.902172","192.168.1.23","se.rv.er","TCP","54","56474 > 443 [ACK] Seq=1 Ack=1 Win=131328 Len=0"
"22152","1090.953562","192.168.1.23","se.rv.er","TLSv1.2","245","Client Hello"
"22153","1090.972912","se.rv.er","192.168.1.23","TLSv1.2","1514","Server Hello"
"22154","1090.972912","se.rv.er","192.168.1.23","TLSv1.2","487","Certificate, Server Key Exchange, Certificate Request, Server Hello Done"
"22155","1090.973017","192.168.1.23","se.rv.er","TCP","54","56474 > 443 [ACK] Seq=192 Ack=1894 Win=131328 Len=0"
"22156","1090.987254","192.168.1.23","se.rv.er","TLSv1.2","869","Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message"
"22157","1091.005783","se.rv.er","192.168.1.23","TLSv1.2","61","Alert (Level: Fatal, Description: Unknown CA)"
"22158","1091.005783","se.rv.er","192.168.1.23","TCP","54","443 > 56474 [FIN, ACK] Seq=1901 Ack=1007 Win=64128 Len=0"
"22159","1091.005898","192.168.1.23","se.rv.er","TCP","54","56474 > 443 [ACK] Seq=1007 Ack=1902 Win=131328 Len=0"
"22160","1091.009858","192.168.1.23","se.rv.er","TCP","54","56474 > 443 [FIN, ACK] Seq=1007 Ack=1902 Win=131328 Len=0"
"22161","1091.028144","se.rv.er","192.168.1.23","TCP","54","443 > 56474 [ACK] Seq=1902 Ack=1008 Win=64128 Len=0"
right. so The client sends certificate (or not) and it gets "22157","1091.005783","se.rv.er","192.168.1.23","TLSv1.2","61","Alert (Level: Fatal, Description: Unknown CA)"
from that server and that kills the session. That happens before client executes the validation logic of the server cert.
So the handshake fails because server fails to validate and kills the session.
Description
I am unable to establish a successful connection to a server which presents a self-signed certificate when using a client certificate with
HttpClientHandler
andHttpClient
. (Please see the code in the Reproduction Steps section).ServerCertificateCustomValidationCallback
is never called.I can successfully connect to the server using the same pfx file with:
curl --insecure --cert-type P12 --cert xxx.pfx:XXX -X GET https://test.example.com:443/aaa
Reproduction Steps
Expected behavior
Objective 1: ServerCertificateCustomValidationCallback is called Objective 2: Connection can be established
Actual behavior
Windows 10 and 11:
Linux (e.g. Ubuntu 18.04.1 LTS)
Regression?
Unsure
Known Workarounds
Not known
Configuration
.net6.0 Linux and Windows, x86 and x64
Other information
Similar but different? https://github.com/dotnet/runtime/issues/20671