.net 5 rc2 send mail by smtp with ssl/tls issue

azraelrabbit commented 3 years ago

Describe the bug

when publish an asp.net core project by .net 5.0.100-rc.2.20479.15 , i use blow command to publish an single file . dotnet publish -c Release -r linux-x64 /p:PublishSingleFile=true

then deploy to ubuntu 18.04 x64 environment, and run as service.

in my application use mailkit to send mail by SMTP with port 465

then i get an error : An error occurred while attempting to establish an SSL or TLS connection.

i try the all steps in : https://github.com/jstedfast/MailKit/blob/master/FAQ.md#SslHandshakeException but the issue always there. if i fallback .net sdk to 3.1 then publish for linux-x64 , the mail could send out .

then i found the finally called methods is :: and the code found at https://github.com/jstedfast/MailKit/blob/ec12517d88f659acde4f81bea0e943b08883c57f/MailKit/Net/Smtp/SmtpClient.cs#L1206


                var ssl = new SslStream (stream, false, ValidateRemoteCertificate);

                try {
                    if (doAsync) {
                        await ssl.AuthenticateAsClientAsync (host, ClientCertificates, SslProtocols, CheckCertificateRevocation).ConfigureAwait (false);
                    } else {
#if NETSTANDARD1_3 || NETSTANDARD1_6
                        ssl.AuthenticateAsClientAsync (host, ClientCertificates, SslProtocols, CheckCertificateRevocation).GetAwaiter ().GetResult ();
#else
                        ssl.AuthenticateAsClient (host, ClientCertificates, SslProtocols, CheckCertificateRevocation);
#endif
                    }
                } catch (Exception ex) {
                    ssl.Dispose ();

                    throw SslHandshakeException.Create (this, ex, false);
                }

when calling AuthenticateAsClient method , the exception throwed.

pls help!

To Reproduce

Exceptions (if any)

An error occurred while attempting to establish an SSL or TLS connection.

This usually means that the SSL certificate presented by the server is not trusted by the system for one or more of the following reasons:

The server is using a self-signed certificate which cannot be verified.
The local system is missing a Root or Intermediate certificate needed to verify the server's certificate.
A Certificate Authority CRL server for one or more of the certificates in the chain is temporarily unavailable.
The certificate presented by the server is expired or invalid.

Another possibility is that you are trying to connect to a port which does not support SSL/TLS.

It is also possible that the set of SSL/TLS protocols supported by the client and server do not match.

See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#SslHandshakeException for possible solutions.

at MailKit.Net.Smtp.SmtpClient.ConnectAsync(String host, Int32 port, SecureSocketOptions options, Boolean doAsync, CancellationToken cancellationToken) at testmail5.MailHelper.SendMailAsync(String toMail, String toName, String messageContent, String subject)

Further technical details

ASP.NET Core version : 5.0.100-rc.2.20479.15
Include the output of dotnet --info .NET SDK (reflecting any global.json): Version: 5.0.100-rc.2.20479.15 Commit: da7dfa8840

Runtime Environment: OS Name: Windows OS Version: 10.0.20185 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\5.0.100-rc.2.20479.15\

Host (useful for support): Version: 5.0.0-rc.2.20475.5 Commit: c5a3f49c88

.NET SDKs installed: 2.2.207 [C:\Program Files\dotnet\sdk] 2.2.402 [C:\Program Files\dotnet\sdk] 3.0.100-preview8-013656 [C:\Program Files\dotnet\sdk] 3.0.100 [C:\Program Files\dotnet\sdk] 3.1.201 [C:\Program Files\dotnet\sdk] 3.1.403 [C:\Program Files\dotnet\sdk] 5.0.100-rc.1.20454.5 [C:\Program Files\dotnet\sdk] 5.0.100-rc.2.20479.15 [C:\Program Files\dotnet\sdk]

.NET runtimes installed: Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.0.0-preview8.19405.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.0-rc.1.20451.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.0-rc.2.20475.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.0.0-preview8-28405-07 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.0-rc.1.20451.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.0-rc.2.20475.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 3.0.0-preview8-28405-07 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.0-rc.1.20452.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.0-rc.2.20475.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

The IDE (VS / VS Code/ VS4Mac) you're running on, and it's version visual studio 2019 preview 16.7.7

azraelrabbit commented 3 years ago

and there is more exception :


-------------------------------1
   at MailKit.Net.Smtp.SmtpClient.ConnectAsync(String host, Int32 port, SecureSocketOptions options, Boolean doAsync, CancellationToken cancellationToken)
   at testmail5.MailHelper.SendMailAsync(String toMail, String toName, String messageContent, String subject) in /mnt/c/Users/rock_/source/repos/testmail5/MailHelper.cs:line 101
--------------------------------2
Authentication failed, see inner exception.|   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at MailKit.Net.Smtp.SmtpClient.ConnectAsync(String host, Int32 port, SecureSocketOptions options, Boolean doAsync, CancellationToken cancellationToken)
--------------------------------3
SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.|   at Interop.OpenSsl.DoSslHandshake(SafeSslHandle context, ReadOnlySpan`1 input, Byte[]& sendBuf, Int32& sendCount)
   at System.Net.Security.SslStreamPal.HandshakeInternal(SafeFreeCredentials credential, SafeDeleteSslContext& context, ReadOnlySpan`1 inputBuffer, Byte[]& outputBuffer, SslAuthenticationOptions sslAuthenticationOptions)
--------------------------------4
error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure|

Dotnet-GitSync-Bot commented 3 years ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

ghost commented 3 years ago

Tagging subscribers to this area: @dotnet/ncl See info in area-owners.md if you want to be subscribed.

wfurt commented 3 years ago

can you do packet capture? Do you know if the server supports TLS1.2 (or above?) Could be https://docs.microsoft.com/en-us/dotnet/core/compatibility/cryptography#default-tls-cipher-suites-for-net-on-linux

azraelrabbit commented 3 years ago

can you do packet capture? Do you know if the server supports TLS1.2 (or above?) Could be https://docs.microsoft.com/en-us/dotnet/core/compatibility/cryptography#default-tls-cipher-suites-for-net-on-linux

use this command : openssl ciphers -v | awk '{print $2}' | sort | uniq the result: SSLv3 TLSv1 TLSv1.2 TLSv1.3

and i do the stuf as https://docs.microsoft.com/en-us/dotnet/core/compatibility/cryptography#default-tls-cipher-suites-for-net-on-linux , the error still exists

wfurt commented 3 years ago

packet capture from 3.1 and 5.x??? What ciphers did you add?

azraelrabbit commented 3 years ago

i test my application in net 5.0 rc1 rc2 and release version in ubuntu 18.04/ubuntu 20.04 , the same error . and in centos 8 work fine.

then i publish my application by .net core 3.0/3.1, ubuntu/centos work fine .

then i change the openssl config in ubuntu 18/20, i add this to /etc/ssl/openssl.conf , but error still exist.

openssl_conf = default_conf

[default_conf]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
CipherString = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:TLS_RSA_WITH_AES_128_CBC_SHA256
Ciphersuites = TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_CCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_RSA_WITH_AES_128_CBC_SHA256
 Options = ServerPreference,PrioritizeChaCha

i just send mail by mailkit. here is the openssl s_client result for smtp.exmail.qq.com: 465

~$  openssl s_client -connect smtp.exmail.qq.com:465
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Secure Site CA G2
verify return:1
depth=0 C = CN, ST = Guangdong Province, L = Shenzhen, O = Tencent Technology (Shenzhen) Company Limited, OU = R&D, CN = exmail.qq.com
verify return:1
---
Certificate chain
 0 s:C = CN, ST = Guangdong Province, L = Shenzhen, O = Tencent Technology (Shenzhen) Company Limited, OU = R&D, CN = exmail.qq.com
   i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Secure Site CA G2
 1 s:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Secure Site CA G2
   i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGZDCCBUygAwIBAgIQDW9FDXfNqUJqMhJjolgyzjANBgkqhkiG9w0BAQsFADBb
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMRowGAYDVQQDExFTZWN1cmUgU2l0ZSBDQSBHMjAeFw0y
MDA3MDgwMDAwMDBaFw0yMTA3MTMxMjAwMDBaMIGbMQswCQYDVQQGEwJDTjEbMBkG
A1UECBMSR3Vhbmdkb25nIFByb3ZpbmNlMREwDwYDVQQHEwhTaGVuemhlbjE2MDQG
A1UEChMtVGVuY2VudCBUZWNobm9sb2d5IChTaGVuemhlbikgQ29tcGFueSBMaW1p
dGVkMQwwCgYDVQQLDANSJkQxFjAUBgNVBAMTDWV4bWFpbC5xcS5jb20wggEiMA0G
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCFPYLaZbdMmz/MgjuD69gi8V7DGFD9
nb3zcRoFvh13l1FQTmcoVgpPo/1sxMuOtMIst8YZ97+gfw+z+5CzFWPZkLSan3v7
/Y7fdLE30D8on19uKmv968HsrZhNCfdOI55bhElklXQTUJvkDqVYxGZ69qUcHydm
Z/Y94ZLJG63JDCuDkx7LX2OlW0xbps1IcIKUFG+/HhFckrI5+mlOdXLdXifrn4ur
VyLr9sPQtEJJHewA0Z9F/oqLIxz68bx823RC0zt1poRgg+newyb4qvwbBnNWz33r
/mcNIsjihEU4XnQxW3BlKTBMHGPd5E29g41X7qI+9Ui4i/NE9SQCl2FZAgMBAAGj
ggLhMIIC3TAfBgNVHSMEGDAWgBTEEX6IQIbCQb9l8xrhtFNAo6vsfTAdBgNVHQ4E
FgQU6jTiJoP1z8W4rjRRr92cnfW8rIkwKQYDVR0RBCIwIIINZXhtYWlsLnFxLmNv
bYIPKi5leG1haWwucXEuY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwbwYDVR0fBGgwZjAxoC+gLYYraHR0cDovL2NybDMu
ZGlnaWNlcnQuY29tL1NlY3VyZVNpdGVDQUcyLmNybDAxoC+gLYYraHR0cDovL2Ny
bDQuZGlnaWNlcnQuY29tL1NlY3VyZVNpdGVDQUcyLmNybDBMBgNVHSAERTBDMDcG
CWCGSAGG/WwBATAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy5kaWdpY2VydC5j
b20vQ1BTMAgGBmeBDAECAjBsBggrBgEFBQcBAQRgMF4wIQYIKwYBBQUHMAGGFWh0
dHA6Ly9vY3NwLmRjb2NzcC5jbjA5BggrBgEFBQcwAoYtaHR0cDovL2NybC5kaWdp
Y2VydC1jbi5jb20vU2VjdXJlU2l0ZUNBRzIuY3J0MAwGA1UdEwEB/wQCMAAwggEE
BgorBgEEAdZ5AgQCBIH1BIHyAPAAdgD2XJQv0XcwIhRUGAgwlFaO400TGTO/3wwv
IAvMTvFk4wAAAXMtuZeFAAAEAwBHMEUCIQCE1hfBf7k/g/HyBIwEqSHlrrtfXqgx
LsA9TQP6LhLekgIgOPkm2EueXZQ+R81fRk1WQ3f+WVHw7bTPaLCiPB51R5gAdgBc
3EOS/uarRUSxXprUVuYQN/vV+kfcoXOUsl7m9scOygAAAXMtuZe2AAAEAwBHMEUC
IQDu+GT0XNuFy0idt4etuRmFkouCO26VLhCcRI/wEyJlPwIgThxtaepxoRxsszEX
PLJsVEzioe0ADk+vhjUFa6nLd30wDQYJKoZIhvcNAQELBQADggEBAAsxDAfNSnPl
VhWfYmIGwHI1qU057UyPtYP8w6b3QmyC7bSfctvhv7STTNU8tw2JZUcP7L8k19Gu
SjpvcgtmnhOWAEyAxU2PtK+/OAoD+1ihZGycmOV0PddGEI+056ianREzs0XZU9Qy
QPA98ysrttReOhTmgccyen+X4C92qVCGpw8xLlZnL5hA51o+1L12JTYyqRbikSSy
00n68awNvP/nZ6BQCEY1f8ig4fyvmcV88eY4K7h8RNrG/x1fnr73z2y1zeyX5Wvv
yBKljhOF5e33N8Ak5LBSsC2Msly/8+rFChxGlqqZtdRq8Fgqw2FDqFoockcOs3Cu
E/mqkwxlPhs=
-----END CERTIFICATE-----
subject=C = CN, ST = Guangdong Province, L = Shenzhen, O = Tencent Technology (Shenzhen) Company Limited, OU = R&D, CN = exmail.qq.com

issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Secure Site CA G2

---
No client certificate CA names sent
---
SSL handshake has read 3313 bytes and written 668 bytes
Verification: OK
---
New, TLSv1.2, Cipher is AES128-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES128-SHA256
    Session-ID: 9A4AB9327F69A311B69BCEDCED6B74A3645D65E4D71AD669CCBC86AAA7F11FC9
    Session-ID-ctx:
    Master-Key: F650A960C61686ADFDDD01B87EAC6C4C4D400094653E2B2326EA6907738A27D425F9A20FA532F34DB67904444E3A2806
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 1200 (seconds)
    TLS session ticket:
    0000 - 5b 0a 1e a8 1a 5b 22 4a-5e f3 80 48 9f 7e 41 c8   [....["J^..H.~A.
    0010 - 37 28 1c 34 c0 f9 97 d3-f3 22 05 e5 d9 10 ec f4   7(.4....."......
    0020 - 6d b5 b7 62 d1 49 be 87-3b e6 27 29 e5 33 81 60   m..b.I..;.').3.`
    0030 - 81 74 b3 bb ec f4 63 01-ae c4 42 64 c1 ee b8 d6   .t....c...Bd....
    0040 - 3d ec f3 58 e3 9f 2d b5-69 dd e8 97 a6 84 21 70   =..X..-.i.....!p
    0050 - 61 e5 12 ba 7c eb 15 a8-77 06 ce cd 71 96 35 91   a...|...w...q.5.
    0060 - b6 70 16 23 b4 c8 f7 eb-dd 3c 1d eb 47 96 f5 61   .p.#.....<..G..a
    0070 - bd f2 30 2a ed d8 18 aa-21 20 44 98 38 55 1e ef   ..0*....! D.8U..
    0080 - 85 5f 2e 83 04 02 60 1a-bb c5 96 ae a1 5b 3b e5   ._....`......[;.
    0090 - 06 3f 23 d4 f1 1f 72 62-94 10 99 7d bf a1 9b 12   .?#...rb...}....
    00a0 - 05 2e c3 d0 76 93 65 06-a1 b8 ee 60 42 5c 67 49   ....v.e....`B\gI

    Start Time: 1605072707
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---
220 smtp.qq.com Esmtp QQ Mail Server

azraelrabbit commented 3 years ago

and here is my send mail code :

using (var client = new SmtpClient())
                {
                    var useSsl =true;

                    client.ServerCertificateValidationCallback = (sender, certificate, chain, errors) => true;

                    client.CheckCertificateRevocation = false;

                    client.SslProtocols = SslProtocols.Ssl3 | SslProtocols.Tls | SslProtocols.Tls11 | SslProtocols.Tls12 | SslProtocols.Tls13;

                    await client.ConnectAsync("smtp.exmail.qq.com", 465, useSsl);

                    // Note: since we don't have an OAuth2 token, disable
                    // the XOAUTH2 authentication mechanism.
                    client.AuthenticationMechanisms.Remove("XOAUTH2");

                    client.Authenticated += Client_Authenticated;
                    // Note: only needed if the SMTP server requires authentication

                    await client.AuthenticateAsync(G.MailAccount, G.Passwd);
                    await client.SendAsync(mailMessage);

                    await client.DisconnectAsync(true);
                }

wfurt commented 3 years ago

This is little bit more complicated. I did SSL scan with https://www.immuniweb.com/ssl/?id=scwOOemq and the site come up with with very short list of supported ciphers (and C grade) However, the reported TLS_RSA_WITH_AES_128_CBC_SHA256 (and the string you added) is cipher name from RFC, not OpenSSL name. To get later, you can run "openssl ciphers" or search documentation. The name you need to addd is "AES128-SHA256". With that, I was able to connect to the site using SslStream with .NET 5 on Ubuntu 18.04. (and it was failing before)

Note that merging with the existing config is little bit tricky. The openssl_conf must be at beginning and and the other sections after initial assignments. (I put them to the bottom of the file). You can verify that with Wireshark and you should see cipher 0x003c in ClienHello ciphers.

Alternatively as experiment you can create fresh file like:

openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_config

[ssl_config]
system_default = tls_defaults

[tls_defaults]
CipherString = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:AES128-SHA256

and set OPENSSL_CONF to point at it e.g. export OPENSSL_CONF=/tmp/openssl.cnf

azraelrabbit commented 3 years ago

thanks for your reply. but i add the this to the bottom of the /etc/ssl/openssl.cnf, like this:

then i try to run my application again, i get the same error as before:

and my system is ubuntu 18.04.4 lts openssl version is 1.1.1

Do I need to do other settings to make openssl.cnf take effect?

azraelrabbit commented 3 years ago

@wfurt thank you very much.

i finally get the openssl.cnf working .

as your mention , i copied the ciphersString from centos 8 and add those content before [ new_oids ] in /etc/ssl/openssl.cnf

and re-run my test application, it's working fine.

thanks a lot.

like this:

ps : this config is worked for me :

openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_config

[ssl_config]
system_default = tls_defaults

[tls_defaults]
CipherString = @SECLEVEL=2:kEECDH:kRSA:kEDH:kPSK:kDHEPSK:kECDHEPSK:-aDSS:-3DES:!DES:!RC4:!RC2:!IDEA:-SEED:!eNULL:!aNULL:!MD5:-SHA384:-CAMELLIA:-ARIA:-AESCCM8
Ciphersuites = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256
MinProtocol = TLSv1.2

ps again: if i add those content in bottom of the /etc/ssl/openssl.cnf it does not work.

wfurt commented 3 years ago

I'm glad you got it working. Perhaps coping whole config may be best option for anybody who hit this in the future.

ChrisIsidora commented 3 years ago

I'm having the same issue. Upgraded from .NET 3.1 to .NET 5 and suddenly I get the above error. I copied config above using docker and added environment to point to the new file but still no luck. Is there something I'm missing?

wfurt commented 3 years ago

Do packet5 capture and make sure you see difference in the cipher list. Run the ssl scanner or use openssl to figure out what cipher list your server supports @ChrisIsidora.

ChrisIsidora commented 3 years ago

@wfurt I ran the ssl scanner and also the openssl s_client command. The ciphers supported by the server are also in this list: https://docs.microsoft.com/en-us/dotnet/core/compatibility/cryptography#default-tls-cipher-suites-for-net-on-linux. The openssl s_client command runs successfully verification and handshake ok and connected over TLSv1.2 using ECDHE-RSA-AES256-GCM-SHA384 as cipher. I'm at a loss at this moment. I also tried setting the CipherSuitePolicy on the SocketsHttpHandler but I get the same error.

wfurt commented 3 years ago

There could be other reasons as well. At this point I think you should focus on the Cipher list in ClientHello. Do you see changes there to reflect code changes? If not, something is not right. If you see ciphers would would expect and the handshake still fails it can perhaps be something else and we would need more data to triage.

ChrisIsidora commented 3 years ago

@wfurt How can I check that in the ClientHello? I enabled all ciphers (ALL:eNULL:aNULL) in the cipherstring for testing and I'm still getting the same issue. Here is my openssl s_client output: https://pastebin.com/Xar1DGAT

wfurt commented 3 years ago

Do packet capture with Wireshark. (or use tcpdump and open it on your workstation with Wireshark. Find ClientHello message and unroll "Cipher Suites" section. Wireshark will should you count as well as actual list of ciphers offered by client) Note that you need to capture the Hello from .NET app, not from openssl. e.g. you need to verify that your changes are visible when SslStream reads the system defaults. Also the example you posted connects to port 443. I'm not sure how that relates to email.

ChrisIsidora commented 3 years ago

@wfurt This had nothing to do with email, but has roughly the same situation, I'm calling an API which requires authentication via ClientCertificate, using httpclient from an asp.net linux container (buster-slim). This worked fine using asp.net core .NET 3.1 (buster-slim) when we upgraded to .NET 5 (buster-slim) this broke and is giving us that same error message as this issue. I would be glad to open a new issue if it really doesn't belong here

wfurt commented 3 years ago

ok, that was not quite clear. Why don't we leave it here for now unless we conclude that the cipher list is not root cause. You may share openssl.conf from the container but I think seeing the handshake is essential. Also make sure it is readable by your app and read notes above how this is sensitive to placement.

ClimberBear commented 3 years ago

I see this issue has been closed with no real solution and/or way to diagnose. The question, in short, is all of us have sites that works fine connecting from other OS (Windows, Mac in my case) or connecting from previous Net Core versions (3.1 in my case). But the combination with Net5 and Linux/Ubuntu fails. The error comes from OpenSSL libraries, but if we test with openssl_client (or curl) work fine. How can we help to diagnose this better? BTW, I have also tried all combinations about cipherstring in openssl.cnf, with no change. I'm suspecting that Net5 does not honor the cipherstring, and is using something hardcoded here.

NOTE: I have open also this question: https://stackoverflow.com/questions/65368193/cannot-make-work-net5-with-ubuntu-20-04-openssl-connection-problem

UPDATE: I suspected that Net5 was not honoring the CipherStrings configured, and I created a test. My suspect was (or appears) wrong. See the test in the stack overflow question

wfurt commented 3 years ago

can you post your openssl.con @ClimberBear and either post address of the server or do ssl scan to see what ciphers and versions it support?

ClimberBear commented 3 years ago

can you post your openssl.con @ClimberBear and either post address of the server or do ssl scan to see what ciphers and versions it support?

The URL is Test URL

This is a console program to test:

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Security;
using System.Threading.Tasks;

namespace BugSSL
{
    class Program
    {
        static void Main(string[] args)
        {
            Curl("https://www.boe.es/diario_boe/xml.php?id=BOE-S-20201216");
        }

        static void Curl(string url)
        {
            HttpWebRequest request;
            HttpWebResponse response;

            ServicePointManager.SecurityProtocol = SecurityProtocolType.SystemDefault;

            try
            {
                Console.WriteLine("START-------------------------------------------------");
                Console.WriteLine($"Getting URL ${url}");
                request = WebRequest.CreateHttp(url);
                request.Method = "GET";

                request.AllowAutoRedirect = true;
                request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
                request.Headers.Add(HttpRequestHeader.UserAgent, "SSLBugTest/0.0.0");
                using (response = (HttpWebResponse)request.GetResponse())
                {
                    Console.WriteLine($"Response status: {response.StatusCode} {response.StatusDescription}");
                    Console.WriteLine("Response headers");
                    foreach(string header in response.Headers)
                    {
                        Console.WriteLine($"    {header}: {response.GetResponseHeader(header)}");
                    }
                    Console.WriteLine($"Content-Type: {response.ContentType}");
                    Console.WriteLine($"Content-Length: {response.ContentLength}");
                }

                Console.WriteLine("END---------------------------------------------------");

            }
            catch (Exception e)
            {
                TextWriter stderr = Console.Error;
                stderr.WriteLine($"Error processing {url}. Error: {e.Message}");
                stderr.WriteLine(e.StackTrace);
                while(e.InnerException!=null)
                {
                    e = e.InnerException;
                    Console.WriteLine($"Inner exception: {e.Message}");
                    stderr.WriteLine(e.StackTrace);
                }
            }

        }
    }
}

My openssl.cnf (but, remember, that I have already tried with minversion1 -- see commented lines)

#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#

# Note that you can include other files from the main configuration
# file using the .include directive.
#.include filename

# This definition stops the following lines choking if HOME isn't
# defined.
HOME            = .

# CipherString = DEFAULT:@SECLEVEL=1
# CipherString = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:AES128-SHA256
openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_config

[ssl_config]
system_default = tls_defaults

[tls_defaults]
CipherString = @SECLEVEL=2:kEECDH:kRSA:kEDH:kPSK:kDHEPSK:kECDHEPSK:-aDSS:-3DES:!DES:!RC4:!RC2:!IDEA:-SEED:!eNULL:!aNULL:!MD5:-SHA384:-CAMELLIA:-ARIA:-AESCCM8
Ciphersuites = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256
MinProtocol = TLSv1.2

# Extra OBJECT IDENTIFIER info:
#oid_file       = $ENV::HOME/.oid
oid_section     = new_oids

# To use this configuration file with the "-extfile" option of the
# "openssl x509" utility, name here the section containing the
# X.509v3 extensions to use:
# extensions        =
# (Alternatively, use a configuration file that has only
# X.509v3 extensions in its main [= default] section.)

[ new_oids ]

# We can add new OIDs in here for use by 'ca', 'req' and 'ts'.
# Add a simple OID like this:
# testoid1=1.2.3.4
# Or use config file substitution like this:
# testoid2=${testoid1}.5.6

# Policies used by the TSA examples.
tsa_policy1 = 1.2.3.4.1
tsa_policy2 = 1.2.3.4.5.6
tsa_policy3 = 1.2.3.4.5.7

####################################################################
[ ca ]
default_ca  = CA_default        # The default ca section

####################################################################
[ CA_default ]

dir     = ./demoCA      # Where everything is kept
certs       = $dir/certs        # Where the issued certs are kept
crl_dir     = $dir/crl      # Where the issued crl are kept
database    = $dir/index.txt    # database index file.
#unique_subject = no            # Set to 'no' to allow creation of
                    # several certs with same subject.
new_certs_dir   = $dir/newcerts     # default place for new certs.

certificate = $dir/cacert.pem   # The CA certificate
serial      = $dir/serial       # The current serial number
crlnumber   = $dir/crlnumber    # the current crl number
                    # must be commented out to leave a V1 CRL
crl     = $dir/crl.pem      # The current CRL
private_key = $dir/private/cakey.pem# The private key

x509_extensions = usr_cert      # The extensions to add to the cert

# Comment out the following two lines for the "traditional"
# (and highly broken) format.
name_opt    = ca_default        # Subject Name options
cert_opt    = ca_default        # Certificate field options

# Extension copying option: use with caution.
# copy_extensions = copy

# Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs
# so this is commented out by default to leave a V1 CRL.
# crlnumber must also be commented out to leave a V1 CRL.
# crl_extensions    = crl_ext

default_days    = 365           # how long to certify for
default_crl_days= 30            # how long before next CRL
default_md  = default       # use public key default MD
preserve    = no            # keep passed DN ordering

# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy      = policy_match

# For the CA policy
[ policy_match ]
countryName     = match
stateOrProvinceName = match
organizationName    = match
organizationalUnitName  = optional
commonName      = supplied
emailAddress        = optional

# For the 'anything' policy
# At this point in time, you must list all acceptable 'object'
# types.
[ policy_anything ]
countryName     = optional
stateOrProvinceName = optional
localityName        = optional
organizationName    = optional
organizationalUnitName  = optional
commonName      = supplied
emailAddress        = optional

####################################################################
[ req ]
default_bits        = 2048
default_keyfile     = privkey.pem
distinguished_name  = req_distinguished_name
attributes      = req_attributes
x509_extensions = v3_ca # The extensions to add to the self signed cert

# Passwords for private keys if not present they will be prompted for
# input_password = secret
# output_password = secret

# This sets a mask for permitted string types. There are several options.
# default: PrintableString, T61String, BMPString.
# pkix   : PrintableString, BMPString (PKIX recommendation before 2004)
# utf8only: only UTF8Strings (PKIX recommendation after 2004).
# nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings).
# MASK:XXXX a literal mask value.
# WARNING: ancient versions of Netscape crash on BMPStrings or UTF8Strings.
string_mask = utf8only

# req_extensions = v3_req # The extensions to add to a certificate request

[ req_distinguished_name ]
countryName         = Country Name (2 letter code)
countryName_default     = AU
countryName_min         = 2
countryName_max         = 2

stateOrProvinceName     = State or Province Name (full name)
stateOrProvinceName_default = Some-State

localityName            = Locality Name (eg, city)

0.organizationName      = Organization Name (eg, company)
0.organizationName_default  = Internet Widgits Pty Ltd

# we can do this but it is not needed normally :-)
#1.organizationName     = Second Organization Name (eg, company)
#1.organizationName_default = World Wide Web Pty Ltd

organizationalUnitName      = Organizational Unit Name (eg, section)
#organizationalUnitName_default =

commonName          = Common Name (e.g. server FQDN or YOUR name)
commonName_max          = 64

emailAddress            = Email Address
emailAddress_max        = 64

# SET-ex3           = SET extension number 3

[ req_attributes ]
challengePassword       = A challenge password
challengePassword_min       = 4
challengePassword_max       = 20

unstructuredName        = An optional company name

[ usr_cert ]

# These extensions are added when 'ca' signs a request.

# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.

basicConstraints=CA:FALSE

# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.

# This is OK for an SSL server.
# nsCertType            = server

# For an object signing certificate this would be used.
# nsCertType = objsign

# For normal client use this is typical
# nsCertType = client, email

# and for everything including object signing:
# nsCertType = client, email, objsign

# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment

# This will be displayed in Netscape's comment listbox.
nsComment           = "OpenSSL Generated Certificate"

# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer

# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move

# Copy subject details
# issuerAltName=issuer:copy

#nsCaRevocationUrl      = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName

# This is required for TSA certificates.
# extendedKeyUsage = critical,timeStamping

[ v3_req ]

# Extensions to add to a certificate request

basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment

[ v3_ca ]

# Extensions for a typical CA

# PKIX recommendation.

subjectKeyIdentifier=hash

authorityKeyIdentifier=keyid:always,issuer

basicConstraints = critical,CA:true

# Key usage: this is typical for a CA certificate. However since it will
# prevent it being used as an test self-signed certificate it is best
# left out by default.
# keyUsage = cRLSign, keyCertSign

# Some might want this also
# nsCertType = sslCA, emailCA

# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy

# DER hex encoding of an extension: beware experts only!
# obj=DER:02:03
# Where 'obj' is a standard or added object
# You can even override a supported extension:
# basicConstraints= critical, DER:30:03:01:01:FF

[ crl_ext ]

# CRL extensions.
# Only issuerAltName and authorityKeyIdentifier make any sense in a CRL.

# issuerAltName=issuer:copy
authorityKeyIdentifier=keyid:always

[ proxy_cert_ext ]
# These extensions should be added when creating a proxy certificate

# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.

basicConstraints=CA:FALSE

# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.

# This is OK for an SSL server.
# nsCertType            = server

# For an object signing certificate this would be used.
# nsCertType = objsign

# For normal client use this is typical
# nsCertType = client, email

# and for everything including object signing:
# nsCertType = client, email, objsign

# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment

# This will be displayed in Netscape's comment listbox.
nsComment           = "OpenSSL Generated Certificate"

# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer

# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move

# Copy subject details
# issuerAltName=issuer:copy

#nsCaRevocationUrl      = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName

# This really needs to be in place for it to be a proxy certificate.
proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo

####################################################################
[ tsa ]

default_tsa = tsa_config1   # the default TSA section

[ tsa_config1 ]

# These are used by the TSA reply generation only.
dir     = ./demoCA      # TSA root directory
serial      = $dir/tsaserial    # The current serial number (mandatory)
crypto_device   = builtin       # OpenSSL engine to use for signing
signer_cert = $dir/tsacert.pem  # The TSA signing certificate
                    # (optional)
certs       = $dir/cacert.pem   # Certificate chain to include in reply
                    # (optional)
signer_key  = $dir/private/tsakey.pem # The TSA private key (optional)
signer_digest  = sha256         # Signing digest to use. (Optional)
default_policy  = tsa_policy1       # Policy if request did not specify it
                    # (optional)
other_policies  = tsa_policy2, tsa_policy3  # acceptable policies (optional)
digests     = sha1, sha256, sha384, sha512  # Acceptable message digests (mandatory)
accuracy    = secs:1, millisecs:500, microsecs:100  # (optional)
clock_precision_digits  = 0 # number of digits after dot. (optional)
ordering        = yes   # Is ordering defined for timestamps?
                # (optional, default: no)
tsa_name        = yes   # Must the TSA name be included in the reply?
                # (optional, default: no)
ess_cert_id_chain   = no    # Must the ESS cert id chain be included?
                # (optional, default: no)
ess_cert_id_alg     = sha1  # algorithm to compute certificate
                # identifier (optional, default: sha1)

ClimberBear commented 3 years ago

NOTICE: As I did not know how to reopen this, I have opened a new one (updating Net5 version). https://github.com/dotnet/runtime/issues/46271

dotnet / runtime