GnuTLS bails with error 110 when opening data connections

[avonheimburg]

When opening passive connections with a client that uses GnuTLS, it errors out with the error 110 "The TLS connection was non-properly terminated." Note that this only occurs for data connections. The initial STARTTLS and subsequent login work fine.

Log below (from FileZilla):

Status: Verbinde mit 127.0.0.1:2121...
Status: Verbindung hergestellt, warte auf Willkommensnachricht...
Status: Initialisiere TLS...
Status: Überprüfe Zertifikat...
Status: TLS-Verbindung hergestellt.
Status: Angemeldet
Status: Empfange Verzeichnisinhalt...
Befehl: PWD
Antwort:    257 "/"
Befehl: TYPE I
Antwort:    200 Binary transfer mode active.
Befehl: PASV
Antwort:    227 Entering Passive Mode (127,0,0,1,217,65).
Befehl: MLSD
Antwort:    150 Opening data connection.
Fehler: GnuTLS-Fehler -110 in gnutls_record_recv: The TLS connection was non-properly terminated.
Status: Server hat die TLS-Verbindung nicht ordnungsgemäß geschlossen
Fehler: Could not read from transfer socket: ECONNABORTED - Verbindung abgebrochen
Antwort:    226 Closing data connection.

[avonheimburg]

Addendum: Perhaps it's related to this old thread:

https://forum.filezilla-project.org/viewtopic.php?f=2&t=8110

[fubar-coder]

That was the reason why I had a customized SslStream implementation in the 1.x versions. It's a well-known incompatibility between SslStream from .NET Framework and GnuTLS. It should be fixed in .NET Core.

[avonheimburg]

Okay, thanks, I'll close this, then.

[fubar-coder]

Reopening due to problem being unresolved on .NET Framework versions older than 4.7

[sebastianbk]

I am still having this issue in .NET Core 2.2, using the latest version of FileZilla on Windows as the client.

Here is my client output:

Status: Resolving address of ftp.criterion.ai
Status: Connecting to 35.210.105.126:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/"
Command:    TYPE I
Response:   200 Binary transfer mode active.
Command:    PASV
Response:   227 Entering Passive Mode (35,210,105,126,39,25).
Command:    MLSD
Response:   150 Opening data connection.
Error:  GnuTLS error -110 in gnutls_record_recv: The TLS connection was non-properly terminated.
Status: Server did not properly shut down TLS connection
Error:  Could not read from transfer socket: ECONNABORTED - Connection aborted
Response:   226 Closing data connection.
Error:  Failed to retrieve directory listing

And here is my server output:

2019-03-24 00:59:58.7488 TRACE 62.199.171.63:61053 SYST  
2019-03-24 00:59:58.7514 TRACE 62.199.171.63:61053 215 UNIX Type: A 
2019-03-24 00:59:58.7908 TRACE 62.199.171.63:61053 FEAT  
2019-03-24 00:59:58.8175 DEBUG 62.199.171.63:61053 211-Extensions supported: 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  AUTH TLS 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  PBSZ 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  PROT 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  HOST 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  LANG en* 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  MDTM 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  MFCT 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  MFF modify;create; 
2019-03-24 00:59:58.8179 DEBUG 62.199.171.63:61053  MFMT 
2019-03-24 00:59:58.8189 DEBUG 62.199.171.63:61053  MLST type*;size*;perm*;modify*;create*; 
2019-03-24 00:59:58.8189 DEBUG 62.199.171.63:61053  UTF8 
2019-03-24 00:59:58.8189 DEBUG 62.199.171.63:61053  EPSV 
2019-03-24 00:59:58.8189 DEBUG 62.199.171.63:61053  EPRT 
2019-03-24 00:59:58.8189 DEBUG 62.199.171.63:61053  REST STREAM 
2019-03-24 00:59:58.8189 DEBUG 62.199.171.63:61053  SIZE 
2019-03-24 00:59:58.8201 TRACE 62.199.171.63:61053 211 END 
2019-03-24 00:59:58.9376 TRACE 62.199.171.63:61053 OPTS UTF8 ON 
2019-03-24 00:59:58.9398 TRACE 62.199.171.63:61053 200 Command okay. 
2019-03-24 00:59:58.9793 TRACE 62.199.171.63:61053 PBSZ 0 
2019-03-24 00:59:58.9817 TRACE 62.199.171.63:61053 200 Protection buffer size set to 0. 
2019-03-24 00:59:59.0223 TRACE 62.199.171.63:61053 PROT P 
2019-03-24 00:59:59.0249 TRACE 62.199.171.63:61053 200 Data channel protection level set to P. 
2019-03-24 00:59:59.0674 TRACE 62.199.171.63:61053 PWD  
2019-03-24 00:59:59.0715 TRACE 62.199.171.63:61053 257 "/" 
2019-03-24 00:59:59.1125 TRACE 62.199.171.63:61053 TYPE I 
2019-03-24 00:59:59.1171 TRACE 62.199.171.63:61053 200 Binary transfer mode active. 
2019-03-24 00:59:59.1573 TRACE 62.199.171.63:61053 PASV  
2019-03-24 00:59:59.1762 TRACE 62.199.171.63:61053 227 Entering Passive Mode (35,210,105,126,39,25). 
2019-03-24 00:59:59.2580 DEBUG 62.199.171.63:61053 Data connection accepted from 62.199.171.63 
2019-03-24 00:59:59.2588 TRACE 62.199.171.63:61053 MLSD  
2019-03-24 00:59:59.2672 DEBUG 62.199.171.63:61053 150 Opening data connection. 
2019-03-24 00:59:59.8058 DEBUG 62.199.171.63:61053 perm=cmpel;type=cdir; . 
2019-03-24 00:59:59.8066 DEBUG 62.199.171.63:61053 perm=cmpdfel;type=dir; failed_cap 
2019-03-24 00:59:59.8066 DEBUG 62.199.171.63:61053 perm=cmpdfel;type=dir; good 
2019-03-24 00:59:59.8066 DEBUG 62.199.171.63:61053 perm=cmpdfel;type=dir; missing_cap 
2019-03-24 00:59:59.8258 DEBUG 62.199.171.63:61053 perm=cdfrw;size=12361;type=file;modify=20190221232820.141;create=20190221232820.141; job_4ade8458-1074-4f23-8aae-2e92cd5be2bf.json 
2019-03-24 00:59:59.8308 TRACE 62.199.171.63:61053 226 Closing data connection. 

It seems like the server is indeed finding the list of files the client is asking for but, for some reason, the connection is being closed by returning the results.

Is there something I can do to help fix this issue?

[sebastianbk]

Hi @fubar-coder: I did a little more testing and it definitely seems as if this problem only pertains to clients based on the GnuTLS library. I tested my server (with a TLS) using Cyberduck, Free FTP and WinSCP and all of them work flawlessly. I can only reproduce the problem with FileZilla.

[fubar-coder]

@sebastianbk The new version of the FTP server library will support custom SslStream implementations. This problem annoys me too.

[sebastianbk]

That sounds great. Is there any ETA on when this will be available? 😊

[fubar-coder]

You can test the current preview version 3.0.0-beta.4 from nuget.org.

EDIT: There is a new interface ISslStreamWrapperFactory and its default implementation is DefaultSslStreamWrapperFactory. Just register your implementation as singleton after your AddFtpServer() call.

[sebastianbk]

Thank you for the tip, @fubar-coder. I tried to make an implemetation of ISslStreamWrapperFactory that makes use of the GnuSslStream class from the GnuSslStream NuGet package. See below:

using System.IO;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using System.Threading;
using System.Threading.Tasks;
using FubarDev.FtpServer.Authentication;

namespace Criterion.Ftp
{
    public class CustomSslStreamWrapperFactory : ISslStreamWrapperFactory
    {
        public async Task<Stream> WrapStreamAsync(
            Stream unencryptedStream,
            bool keepOpen,
            X509Certificate certificate,
            CancellationToken cancellationToken)
        {
            var sslStream = new GnuSslStream(unencryptedStream, keepOpen);
            try
            {
                await sslStream.AuthenticateAsServerAsync(certificate)
                    .ConfigureAwait(false);
            }
            catch
            {
                sslStream.Dispose();
                throw;
            }

            return sslStream;
        }

        public Task CloseStreamAsync(Stream sslStream, CancellationToken cancellationToken)
        {
            if (sslStream is GnuSslStream s)
            {
                s.Close();
            }

            return Task.CompletedTask;
        }
    }
}

Here is my Program.cs where I register my implementation in DI:

using System;
using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.Threading;
using Criterion.Ftp.FileSystem.Gcs;
using FubarDev.FtpServer;
using FubarDev.FtpServer.AccountManagement;
using FubarDev.FtpServer.Authentication;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using NLog.Extensions.Logging;

namespace Criterion.Ftp
{
    class Program
    {
        static void Main(string[] args)
        {
            var services = new ServiceCollection().AddLogging(config => config.SetMinimumLevel(LogLevel.Trace));

#if !DEBUG
            var disableTls = Environment.GetEnvironmentVariable("DISABLE_TLS") ?? "False";
            if (bool.TryParse(disableTls, out var disable))
            {
                if (!disable)
                {
                    var cert = new X509Certificate2("ftp.pfx", Environment.GetEnvironmentVariable("PFX_PASSWORD"));
                    services.Configure<AuthTlsOptions>(cfg => cfg.ServerCertificate = cert);
                }
            }
#endif

            services.Configure<FtpConnectionOptions>(options => options.DefaultEncoding = System.Text.Encoding.UTF8);
            services.Configure<SimplePasvOptions>(options =>
            {
                options.PasvMinPort = 10000;
                options.PasvMaxPort = 10009;
                options.PublicAddress = IPAddress.Parse(Environment.GetEnvironmentVariable("PUBLIC_IP") ?? "127.0.0.1");
            });

            services.AddFtpServer(builder =>
            {
                builder.Services.AddSingleton<IMembershipProvider, CustomMembershipProvider>();
                builder.UseGcsFileSystem();
            });

            services.AddSingleton<ISslStreamWrapperFactory, CustomSslStreamWrapperFactory>();

            // Build the service provider
            using (var serviceProvider = services.BuildServiceProvider())
            {
                var loggerFactory = serviceProvider.GetRequiredService<ILoggerFactory>();
                loggerFactory.AddNLog(new NLogProviderOptions { CaptureMessageTemplates = true, CaptureMessageProperties = true });
                NLog.LogManager.LoadConfiguration("NLog.config");

                try
                {
                    // Initialize the FTP server
                    var ftpServerHost = serviceProvider.GetRequiredService<IFtpServerHost>();

                    // Start the FTP server
                    ftpServerHost.StartAsync(CancellationToken.None).ConfigureAwait(false);

                    Console.WriteLine("The FTP server is running. Press any key to kill the server...");
                    Console.ReadLine();

                    // Stop the FTP server
                    ftpServerHost.StopAsync(CancellationToken.None).ConfigureAwait(false);
                }
                catch (Exception ex)
                {
                    Console.Error.WriteLine(ex);
                }
            }
        }
    }
}

However, I still get an error in FileZilla. See below:

Status: Resolving address of ftp.criterion.ai
Status: Connecting to 35.210.105.126:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Logged in
Status: Retrieving directory listing...
Command:    PWD
Response:   257 "/"
Command:    TYPE I
Response:   200 Binary transfer mode active.
Command:    PASV
Response:   227 Entering Passive Mode (35,210,105,126,39,20).
Command:    MLSD
Response:   150 Opening data connection.
Error:  GnuTLS error -110 in gnutls_record_recv: The TLS connection was non-properly terminated.
Status: Server did not properly shut down TLS connection
Error:  Could not read from transfer socket: ECONNABORTED - Connection aborted
Response:   226 Closing data connection.
Error:  Failed to retrieve directory listing

In other clients (such as Cyberduck and WinSCP), I have no issues at all, even with the new implementation.

I took a look at the implementation of GnuSslStream (see the link below) and it looks like the only difference between that class and the native SslStream is that the SslDirectCall.CloseNotify method is being called when closing the connection. However, even though I am running on Windows, it seems as if that call isn't working correctly. At least, there does not seem to be any difference between using SslStream and GnuSslStream.

https://github.com/UlyssesWu/UniFTP/blob/0a1e75c1b6363fa1cd624202b659b8a25a52042c/GnuSslStream/GnuSslStream.cs#L59

I even tried to remove my reference to the GnuSslStream NuGet package and download the files from the package (GnuSslStream.cs, NativeApi.cs, ReflectUtil.cs and SslDirectCall.cs) instead. I then modified the Close method to make a call to SslDirectCall.CloseNotify even when running on Linux (which the original implementation did not allow for). See below. I then ran my FTP server on a Linux box instead of Windows but that, unfortunately, did not make any difference. I could still connect from Cyberduck, Free FTP and WinSCP but not from FileZilla.

public override void Close()
{
    try
    {
        SslDirectCall.CloseNotify(this);
    }
    finally
    {
        base.Close();
    }
}

So, basically, after having tried a bunch of different things, I still haven't gotten much further. Is there something I did wrong or is there anything I can do to help identifying the root cause of this issue?

[fubar-coder]

It seems that there was still a problem in the library. When your application is a .NET Core app, then everything will automatically work. When you cannot user .NET Core, then you can use the code from the example FTP server with a modified GnuSslStream. Version 3.0.0-beta.5 should be available soon.

[sebastianbk]

I am using .NET Core 2.2 (on Linux) but I still have the issue (even after upgrading to v3.0.0-beta.5 and using the CustomSslStreamWrapperFactory in the sample project).

I use Docker to host my application and my image is based on microsoft/dotnet:2.2-sdk. I guess I will have to submit a bug report to the .NET Core team.

[fubar-coder]

Does the problem exist without the CustomSslStreamWrapperFactory? The default implementation should work now under Linux. (since the latest beta)

[fubar-coder]

IMHO the GnuSslStream should still be buggy under Linux as it's just a Windows only fix for non-netcoreapp projects. I have to fix the sample.

[sebastianbk]

@fubar-coder, you are a hero! Using the latest version (beta.5), I can now access my FTP server (running on Linux) with FileZilla via FTPS over TLS. 😃

I am just using the default implementation.

[sebastianbk]

Thank you so much for all your hard work. Do you have a Patreon site or something similar? Would love to donate to the project.

[fubar-coder]

Changes are in 3.0-rc.1

[BoditraxMike]

I've just hit this today - is the fix in the sample project?

Edit: It's not a blocker, I used the nuget packages instead. Love this library btw - it saved me setting up an FTP server just for one little job :)

[fubar-coder]

It's not in the sample project, it's in the main library.