MailKit.Security.SslHandshakeException misleading error actually due to missing libcurl library

[WhatFreshHellIsThis]

This was a vexing problem that I found a solution for and am posting in case anyone else runs into this issue.

I have a asp.net Core 2.1 project with Mailkit 2.0.5 on Ubuntu 16.04.4 and a Surgemail mail server we host ourselves.

The application is an internal company app that uses IMAP to check for mail and display it in a web page. It worked fine on a Windows server for months. We recently ported our online services to Linux so I installed it on Linux with the .net 2.1 runtime and got the error stack trace in the block below when using IMAP to connect to the mail server.

Because the error specifically states it's a certificate issue I spent a lot of time trying many different things including opening the mail server non SSL port for IMAP and turning off SSL in my code completely, everything worked no matter what I did on my local Windows dev station but not on the Linux server.

When I completely turned off SSL in MailKit and also overrode the certificate check and it still gave the same error about the certificate I knew something more was up.

The resolution turned out to be a missing pre-requisite that was not installed with the .net runtime libcurl.

After running the following command it immediately started working without error:

apt-get install libcurl3

So it seems .net 2.1 runtime installation doesn't actually install all the pre-requisites required for Mailkit.

It's a bit strange that this particular error message comes up when a library is missing and all SSL options are turned off.

I hope this helps someone else in future.

(MailKit.Security.SslHandshakeException: An error occurred while attempting to establish an SSL or TLS connection.

The SSL certificate presented by the server is not trusted by the system for one or more of the following reasons:
1. The server is using a self-signed certificate which cannot be verified.
2. The local system is missing a Root or Intermediate certificate needed to verify the server's certificate.
3. The certificate presented by the server is expired or invalid.

See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#InvalidSslCertificate for possible solutions. ---> System.TypeInitializationException: The type initializer for 'Http' threw an exception. ---> System.TypeInitializationException: The type initializer for 'HttpInitializer' threw an exception. ---> System.DllNotFoundException: Unable to load shared library 'System.Net.Http.Native' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libSystem.Net.Http.Native: cannot open shared object file: No such file or directory
   at Interop.Http.GetSslVersionDescription())

[jstedfast]

MailKit will toggle into SSL mode using the STARTTLS command if it is available unless you use SecureSocketOptions.None in the Connect() method that takes a SecureSocketOptions argument (as opposed to true/false).

Hope that helps.

[WhatFreshHellIsThis]

Thank you, that wasn't really the issue here but that's useful to know.

That error message as I said could possibly benefit by adding that it isn't necessarily any of the things it says but possibly a missing pre-requisite .dll file which might avoid a "wild goose chase" scenario for others in future.
Also, passing false to Connect ssl parameter would seem to intuitively indicate it will in no way try to establish an ssl connection, perhaps there's an opportunity for improvement there.

[PeterHagen]

I seem to have a simular issue. I get the same message on asp.net core 2.1 project in Docker. I use the microsoft/dotnet:2.1-aspnetcore-runtime image (latest), but with older versions I had the same issue. When I run the same code locally (macos without Docker) the exception doesn't appear, and it works fine. libcurl3 is up to date on the Docker image.

I now resolved it by added the ServerCertificateValidationCallback and returning true. Not the best solution, but I'm out of options to get it working.

  client.ServerCertificateValidationCallback = (object sender,
                X509Certificate certificate,
                X509Chain chain,
                SslPolicyErrors sslPolicyErrors) => true;

The certificate is from LetsEncrypt. I also tried to update the root ca certificates, but also no luck. Could it be there is some issue with a missing CA?

[jstedfast]

You'd have to look at the sslPolicyErrors, the certificate chain, etc to know.

[PeterHagen]

I found out that the sslPolicyErrors is a RemoteCertificateChainErrors. That probably has something to do with the ChainStatus, but that part I'll try to debug today

[PeterHagen]

I got some debug information from the chain:

cms_1  | [20-12-2018 21:28:11|14] ServerCertificateValidationCallback X509Certificate Issuer=CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US, Subject=CN=balder.willow-media.nl
cms_1  | [20-12-2018 21:28:11|14] ServerCertificateValidationCallback sslPolicyErrors=RemoteCertificateChainErrors
cms_1  | [20-12-2018 21:28:11|14] ServerCertificateValidationCallback chain=System.Security.Cryptography.X509Certificates.X509Chain ChainStatus=System.Security.Cryptography.X509Certificates.X509ChainStatus[]
cms_1  | [20-12-2018 21:28:11|14] RevocationStatusUnknown - unable to get certificate CRL
cms_1  | [20-12-2018 21:28:11|23] An error occurred while attempting to establish an SSL or TLS connection.
cms_1  | 
cms_1  | The SSL certificate presented by the server is not trusted by the system for one or more of the following reasons:
cms_1  | 1. The server is using a self-signed certificate which cannot be verified.
cms_1  | 2. The local system is missing a Root or Intermediate certificate needed to verify the server's certificate.
cms_1  | 3. The certificate presented by the server is expired or invalid.
cms_1  | 
cms_1  | See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#InvalidSslCertificate for possible solutions.
cms_1  |    at MailKit.Net.Smtp.SmtpClient.ConnectAsync(String host, Int32 port, SecureSocketOptions options, Boolean doAsync, CancellationToken cancellationToken)
cms_1  |    at MailKit.Net.Smtp.SmtpClient.Connect(String host, Int32 port, SecureSocketOptions options, CancellationToken cancellationToken)
cms_1  |    at WillowMedia.Mvc.CertValidationMailkitSmtpService.SendMessage(SmtpSettings smtpSettings, ObjectDisposer disposer, MimeMessage message) in /opt/data/repo/src/WillowMedia.Mvc/Startup.cs:line 102
cms_1  |    at WillowMedia.Common.Smtp.MailkitSmtpService.SendMessage(SmtpSettings smtpSettings, SmtpMessageModel msg)
cms_1  |    at WillowMedia.Cms.Portal.Controllers.CmsTestController.TestSendMail(TestSendEmailModel model)
cms_1  |    at lambda_method(Closure , Object , Object[] )
cms_1  |    at Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(Object target, Object[] parameters)
cms_1  |    at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
cms_1  |    at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeActionMethodAsync()
cms_1  |    at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeNextActionFilterAsync()
cms_1  |    at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context)
cms_1  |    at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
cms_1  |    at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
cms_1  |    at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextExceptionFilterAsync()

This is running with Dotnet core 2.1 in Docker. If I run the same code on a Mac with dotnet 2.1, this doesn't appear.

[20-12-2018 21:37:10|17] ServerCertificateValidationCallback X509Certificate Issuer=CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US, Subject=CN=balder.willow-media.nl
[20-12-2018 21:37:10|17] ServerCertificateValidationCallback sslPolicyErrors=None
[20-12-2018 21:37:10|17] ServerCertificateValidationCallback chain=System.Security.Cryptography.X509Certificates.X509Chain ChainStatus=System.Security.Cryptography.X509Certificates.X509ChainStatus[]

So, I have no clue if this is in any way a mailkit issue, or if something else is going on. Any help is appreciated.

[jstedfast]

MailKit aborts the connection if there are any errors, so sslPolicyErrors being set to something other than None already means this isn't a MailKit bug.

You have to implement a callback method that handles the error and then returns true.

[PeterHagen]

I already thought so, that its not a mailkit bug. Just hoped to get some clues to find the solution. I implemented return true already to get around the problem, but a real fix would be nice. So far I found out its not limited to Docker, but it seems to be a more general linux and dotnet issue.

Thanks for your time, I'll go bother the dotnet team now :D

[Rstar-Yan]

@PeterHagen have you problem resolved?

[darktimes]

@DekeYan I recently found a solution to this. Before calling the client.Connect(), set client.CheckCertificateRevocation to false.

[xtoblizi]

@DekeYan I recently found a solution to this. Before calling the client.Connect(), set client.CheckCertificateRevocation to false.

I can't seem to find the CheckCertificateRevocation property or method on the SmtpClient class

[jstedfast]

It's on the base class, MailService. This property is not available for UWP, though.

[andryushchenko]

I have the same issue on ASP.Net Core 2.2 Docker image mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim On Windows everrything is OK I have this problem with SMTP on smtp.yandex.ru, haven't tested on others. @darktimes solution helps, but it isn't perfect by security reasons

[jstedfast]

The CheckCertificateRevocation option is simply passed to System.Net.Security.SslStream.

If SslStream is failing to check certificate revocation, it means that the CA server is unavailable for CRL requests.

There is no solution that you can do as a client. It MUST be fixed by the CA and/or the server.

[xtoblizi]

Thanks Jeffrey

I have had this fixed by updating my Mailkit nugget package to version 2.5.1 and added the CertificateRevocation property to false, Also i set SSL to false.

Thanks

On Tue, May 28, 2019 at 11:24 AM Jeffrey Stedfast notifications@github.com wrote:

The CheckCertificateRevocation option is simply passed to System.Net.Security.SslStream.

If SslStream is failing to check certificate revocation, it means that the CA server is unavailable for CRL requests.

There is no solution that you can do as a client. It MUST be fixed by the CA and/or the server.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jstedfast/MailKit/issues/735?email_source=notifications&email_token=AHDE4HCJJKZE43DBHWDXJFLPXV2FBA5CNFSM4FJ45OZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWNAJPA#issuecomment-496633020, or mute the thread https://github.com/notifications/unsubscribe-auth/AHDE4HFVRX6DXTUDLUHS5ALPXV2FBANCNFSM4FJ45OZA .

-- Ogbosuka Chris

[msvprogs]

@andryushchenko Same here with ASP.NET Core 2.2.2 Docker image 'microsoft/dotnet:2.2.2-aspnetcore-runtime' and smtp.yandex.com server.

[Vishujai]

See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#InvalidSslCertificate for possible solutions. Please help me..........................................

[Vishujai]

I have prepare a bot in Email automation I have not run See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#InvalidSslCertificate for possible solutions. please reslove the problem

[Vishujai]

@PeterHagen I have prepare a bot in Email automation I have not run See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#InvalidSslCertificate for possible solutions. please reslove the problem

[jstedfast]

@Vishujai check out the docs at https://github.com/jstedfast/MailKit/blob/master/FAQ.md#SslHandshakeException

[Vishujai]

Please share the step by step process in Ui path

[jstedfast]

bool MyCustomServerCertificateValidationCallback (object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
    if (sslPolicyErrors == SslPolicyErrors.None)
        return true;

    var hostName = (string) sender;
    var acceptCertificateDialog = new MyCustomAcceptCertificateDialog(hostName, certificate);

    // Show dialog as a modal dialog and determine if DialogResult = OK.
    if (acceptCertificateDialog.ShowDialog() == DialogResult.OK)
        return true;

    return false;
}

For more documentation on how to create dialogs, see https://docs.microsoft.com/en-us/dotnet/api/system.windows.forms.form.showdialog?view=netcore-3.1

That's all I can do to help you. The rest is up to you to figure out because I don't know Windows.Forms and if you are writing a Windows.Forms app, I'm sure creating a dialog window must be the easiest thing in the world to do compared to the rest of the code.

[haraldkofler]

I encountered the same issue: mailkit has been able to connect to the mailserver (aruba.it) on windows and Linux (ubuntu), but not running inside the docker image. Finally, I found that it works with the "mcr.microsoft.com/dotnet/core/aspnet:3.1-alpine" docker image, which seems to have other openssl settings.

Hope this helps others

[ScholliYT]

@haraldkofler Thanks for the solution. Using 3.1-apline aspnet base image my SMPTClient finally connects to the Mailserver... However I got other issues with Alpine because I'm using some libraries which refuse setup via ldconfig. Therefore, I'm proposing another solution using a separate SMTP Relay Container along with the aspnet Container running Debian/Ubuntu. This allows offloading the Secure connection to the relay Container. The Connection between the dotnet app and the relay container can be made "insecure" (using no encryption).

[tparikka]

I also ran into this after troubleshooting the Connect option. Even after setting useSsl in the connect method to false the client tries to connect with SSL. I had to pass in SecureSocketOptions as instructed here. It just seems like the method is a bit misleading in its behavior.

[jstedfast]

The problem is that it is impossible to map a tri-state option using a bool value.

The useSsl parameter is really only used to specify whether the port is an SSL port or a plain text port.