jstedfast / MailKit

A cross-platform .NET library for IMAP, POP3, and SMTP.
http://www.mimekit.net
MIT License
6.2k stars 822 forks source link

IOException --> SocketException (An existing connection was forcibly closed by the remote host) when using TLS #312

Closed knocte closed 8 years ago

knocte commented 8 years ago

I'm trying to connect to my own postfix server configured on Ubuntu 14.04.

I configured it to disallow any old SSL versions, and to disallow mail rely to non-authenticated users.

I've tested it and it works well from Thunderbird, using STARTTLS (instead of SSL/TLS) and "Normal password" authentication settings, port 587.

However, trying to send mail with MailKit results in this exception.

The code:

            var secureSmtpServerUsername = "test";
            var secureSmtpServerDomainForFromAddress = "@staff.mydomain.com";
            var bodyPart = new TextPart("html"){
                Text = body
            };

            var from = new[] { new MailboxAddress(from_name, secureSmtpServerUsername + secureSmtpServerDomainForFromAddress) };

            var mail = new MimeMessage(from, toList, subject, bodyPart);

            using (var smtpClient = new SmtpClient())
            {
                smtpClient.AuthenticationMechanisms.Remove("XOAUTH2");
                smtpClient.Connect(secureSmtpServerHost, (int)secureSmtpServerPort, true);
                smtpClient.AuthenticationMechanisms.Remove("XOAUTH2");

                smtpClient.Authenticate(secureSmtpServerUsername, secureSmtpServerPassword);
                smtpClient.Send(mail);
            }

The exception happens at the Connect() method, before we had a chance to Authenticate().

Any ideas?

Full ex.ToString():

{System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.AuthenticateAsClient(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation)
   at MailKit.Net.Smtp.SmtpClient.Connect(String host, Int32 port, SecureSocketOptions options, CancellationToken cancellationToken)
   at MailKit.MailService.Connect(Uri uri, CancellationToken cancellationToken)
   at BTCEX.Util.SmtpEmailSender.SendEmail(String[] toemail, String subject, String body, Boolean isBodyHtml, String from_name, String bccAddress) in c:\Users\Andres\Documents\Code\api\api\BTCEX\BTCEX.Util\SmtpEmailSender.cs:line 93
   at BTCEX.Util.SmtpEmailSender.<>c__DisplayClass5.<SendEmailAsync>b__4() in c:\Users\Andres\Documents\Code\api\api\BTCEX\BTCEX.Util\SmtpEmailSender.cs:line 139}
knocte commented 8 years ago

Forgot to say, I'm using these versions of {Mail/Mime}Kit, under Windows 8.1 (MS.NET 4.5):

    <Reference Include="MailKit, Version=1.2.0.0, Culture=neutral, PublicKeyToken=4e064fe7c44a8f1b, processorArchitecture=MSIL">
      <HintPath>..\packages\MailKit.1.2.21\lib\net45\MailKit.dll</HintPath>
      <Private>True</Private>
    </Reference>
    </Reference>
    <Reference Include="MimeKit, Version=1.2.0.0, Culture=neutral, PublicKeyToken=bede1c8a46c66814, processorArchitecture=MSIL">
      <HintPath>..\packages\MimeKit.1.2.22\lib\net45\MimeKit.dll</HintPath>
    </Reference>
knocte commented 8 years ago

I've tried attaching a protocolLogger this way:

var smtpClient = new SmtpClient(new ProtocolLogger("C:\\tmp\\MailKit.log"))

And it creates the MailKit.log file, but it ends up with 0 size :-(

knocte commented 8 years ago

Looking at the logs of postfix is not helpful either, I just see:

Mar 17 07:10:39 postfix postfix/submission/smtpd[18401]: connect from 203185050202.static.ctinets.com[203.185.50.202]
Mar 17 07:10:39 postfix postfix/submission/smtpd[18401]: lost connection after UNKNOWN from 203185050202.static.ctinets.com[203.185.50.202]
Mar 17 07:10:39 postfix postfix/submission/smtpd[18401]: disconnect from 203185050202.static.ctinets.com[203.185.50.202]
knocte commented 8 years ago

If I use:

smtpClient.Connect(secureSmtpServerHost, port:(int)secureSmtpServerPort, options:SecureSocketOptions.StartTls);

I then get a different exception:

{System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
   at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.AuthenticateAsClient(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation)
   at MailKit.Net.Smtp.SmtpClient.Connect(String host, Int32 port, SecureSocketOptions options, CancellationToken cancellationToken)
   at BTCEX.Util.SmtpEmailSender.SendEmail(String[] toemail, String subject, String body, Boolean isBodyHtml, String from_name, String bccAddress) in c:\Users\Andres\Documents\Code\api\api\BTCEX\BTCEX.Util\SmtpEmailSender.cs:line 100
   at BTCEX.Util.SmtpEmailSender.<>c__DisplayClass5.<SendEmailAsync>b__4() in c:\Users\Andres\Documents\Code\api\api\BTCEX\BTCEX.Util\SmtpEmailSender.cs:line 153}
knocte commented 8 years ago

System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.

BTW, the certificate in the email server is fine (otherwise Thunderbird would give a warning about a bad certificate). The email server is even hardened with DKIM&SPF so that emails received from it in Gmail (for example) don't have Phishing or Non-encryption warnings (those annoying, but useful, open red lock icons that Gmail now shows).

jstedfast commented 8 years ago

Hey knocte!

The problem with the Connect (host, 587, true); is that you are trying to connect to a clear-text SMTP port using an SSL-wrapped stream.

As far as the certificate issue, the problem is probably that the Mono cert database does not trust the CA cert that signed your mail server's cert.

You can override the behavior by setting a custom callback on client.ServerCertificateValidationCallback such as:

client.ServerCertificateValidationCallback = () => true;

FWIW, this is the default certificate validation logic: https://github.com/jstedfast/MailKit/blob/master/MailKit/MailService.cs#L253

I wouldn't really recommend that for production code since it accepts all self-signed certs, but it might serve as a starting point for your own custom logic.

jstedfast commented 8 years ago

oh, and make sure to continue using the second Connect() overload that you started using since that gives you more control over what SSL method to use.

The Connect() overload that takes a true/false argument only allows the specification of whether the initial connection should be made using an SslStream or not. If you pass false, it will try to use STARTTLS if the server supports it, but I don't like that API because it's not obvious.

knocte commented 8 years ago

Hey Jeff! thanks for the answer. See inline:

but I don't like that API because it's not obvious.

Agreed.

As far as the certificate issue, the problem is probably that the Mono cert database does not trust the CA cert that signed your mail server's cert.

This cannot be the problem, because, let me remind you, I'm not using Mono currently, but MS.NET ;-) So how do you suggest I could troubleshoot the certificate problem?

jstedfast commented 8 years ago

I'd recommend overriding the callback and seeing what errors it gives.

Remember: MailKit doesn't control the errors or the cert validation process, that is all done by SslStream.

knocte commented 8 years ago

My god, you wouldn't believe what was the problem!

I added the infamous

ServicePointManager.ServerCertificateValidationCallback = delegate(object s, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
    return true;
};

And put a breakpoint on it. Then I found out one of the certificates was signed by AVAST (my fucking antivirus!). Once I disabled my antivirus, sending email started working.

FFS! I'm dying to go back to using Linux where I don't need this kind of shitty software...

Thanks for your help Jeff, always appreciated.

knocte commented 8 years ago

For the record, I guess this is more info about my problem: https://www.avast.com/no-no/faq.php?article=AVKB91

jstedfast commented 8 years ago

Wow, interesting. I had no idea Avast did that...