FTPS not working on Linux since SSL session resumption is not supported

[cluetjen]

FTP OS: Windows 10

FTP Server: FileZilla Server Windows 10

Computer OS: Ubuntu 16.04 (WSL Win 10), Ubuntu 18.04 native

When FluentFTP tries to create a data connection to an FTPS server that requires that the data connection ssl session equals the control session this connection will fail becaus it's a different session.

• It works on windows hosts but not on WSL (bash for windows) or Linux
• Tested with .net core 2.1 and 2.2 preview 3
• It does not matter if implicit or explicit encryption is selected
• These settings are current recommended standard
• While disabling "require TLS session resumption ..." in server would solve the problem, that seems to be not an option. ("hey customer, please turn off standard security features because..." is a little problematic...)

Test code used

    public class FileZillaTests
    {
        [Fact]
        public async Task We_should_be_able_to_connect_and_list_files()
        {
            FtpTrace.LogToFile = "fluentftp.log";
            var client = new FluentFTP.FtpClient("localhost", "test", "test")
            {
                EncryptionMode = FluentFTP.FtpEncryptionMode.Explicit
            };
            client.ValidateCertificate += (context, e) => { e.Accept = true; };

            var files = await client.GetListingAsync("/");

            Assert.NotEmpty(files);
        }
    }

Test result is:

[xUnit.net 00:00:03.8585874]     FileTransfer.Tests.FileZillaTests.We_should_be_able_to_connect_and_list_files [FAIL]
Fehler FileTransfer.Tests.FileZillaTests.We_should_be_able_to_connect_and_list_files
Fehlermeldung:
 System.IO.IOException : Authentication failed because the remote party has closed the transport stream.
Stapelverfolgung:
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Security.SslState.ThrowIfExceptional()
   at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__46_2(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at FluentFTP.FtpSocketStream.ActivateEncryptionAsync(String targethost, X509CertificateCollection clientCerts, SslProtocols sslProtocols)
   at FluentFTP.FtpClient.OpenPassiveDataStreamAsync(FtpDataConnectionType type, String command, Int64 restart)
   at FluentFTP.FtpClient.OpenDataStreamAsync(String command, Int64 restart)
   at FluentFTP.FtpClient.GetListingAsync(String path, FtpListOption options)
   at FileTransfer.Tests.FileZillaTests.We_should_be_able_to_connect_and_list_files() in /mnt/c/workspace/lib-cs-filetransfer/FileTransfer.Tests/FileZillaTests.cs:line 20
--- End of stack trace from previous location where exception was thrown ---

Logs :

# GetListingAsync("/", Auto)

# Connect()
Status:   Connecting to 127.0.0.1:21
Response: 220 Please visit https://filezilla-project.org/
Response: 220-FileZilla Server 0.9.60 beta
Response: 220-written by Tim Kosse (tim.kosse@filezilla-project.org)
Command:  AUTH TLS
Response: 234 Using authentication type TLS
Status:   FTPS Authentication Successful
Status:   Time to activate encryption: 0h 0m 0s.  Total Seconds: 0,8153198.
Command:  USER test
Response: 331 Password required for test
Command:  PASS ***
Response: 230 Logged on
Command:  PBSZ 0
Response: 200 PBSZ=0
Command:  PROT P
Response: 200 Protection level set to P
Command:  FEAT
Response: 211 End
Response: 211-Features:
Response: MDTM
Response: REST STREAM
Response: SIZE
Response: MLST type*;size*;modify*;
Response: MLSD
Response: AUTH SSL
Response: AUTH TLS
Response: PROT
Response: PBSZ
Response: UTF8
Response: CLNT
Response: MFMT
Response: EPSV
Response: EPRT
Status:   Text encoding: System.Text.UTF8Encoding+UTF8EncodingSealed
Command:  OPTS UTF8 ON
Response: 202 UTF8 mode is always enabled. No need to send this command.
Command:  SYST
Response: 215 UNIX emulated by FileZilla
Status:   Auto-detected UNIX listing parser
Command:  TYPE I
Response: 200 Type set to I

# OpenPassiveDataStreamAsync(AutoPassive, "MLSD /", 0)
Command:  EPSV
Response: 229 Entering Extended Passive Mode (|||58695|)
Status:   Connecting to 127.0.0.1:58695
Command:  MLSD /
Response: 150 Opening data channel for directory listing of "/"
Status:   Disposing FtpSocketStream...

FileZilla server logs:

(000010)14.11.2018 22:27:52 - (not logged in) (127.0.0.1)> Connected on port 21, sending welcome message...
(000010)14.11.2018 22:27:52 - (not logged in) (127.0.0.1)> 220-FileZilla Server 0.9.60 beta
(000010)14.11.2018 22:27:52 - (not logged in) (127.0.0.1)> 220-written by Tim Kosse (tim.kosse@filezilla-project.org)
(000010)14.11.2018 22:27:52 - (not logged in) (127.0.0.1)> 220 Please visit https://filezilla-project.org/
(000010)14.11.2018 22:27:52 - (not logged in) (127.0.0.1)> AUTH TLS
(000010)14.11.2018 22:27:52 - (not logged in) (127.0.0.1)> 234 Using authentication type TLS
(000010)14.11.2018 22:27:52 - (not logged in) (127.0.0.1)> TLS connection established
(000010)14.11.2018 22:27:53 - (not logged in) (127.0.0.1)> USER test
(000010)14.11.2018 22:27:53 - (not logged in) (127.0.0.1)> 331 Password required for test
(000010)14.11.2018 22:27:53 - (not logged in) (127.0.0.1)> PASS ****
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 230 Logged on
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> PBSZ 0
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 200 PBSZ=0
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> PROT P
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 200 Protection level set to P
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> FEAT
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 211-Features:
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  MDTM
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  REST STREAM
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  SIZE
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  MLST type*;size*;modify*;
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  MLSD
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  AUTH SSL
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  AUTH TLS
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  PROT
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  PBSZ
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  UTF8
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  CLNT
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  MFMT
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  EPSV
(000010)14.11.2018 22:27:53 - test (127.0.0.1)>  EPRT
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 211 End
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> OPTS UTF8 ON
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 202 UTF8 mode is always enabled. No need to send this command.
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> SYST
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 215 UNIX emulated by FileZilla
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> TYPE I
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 200 Type set to I
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> EPSV
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 229 Entering Extended Passive Mode (|||58695|)
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> MLSD /
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 150 Opening data channel for directory listing of "/"
(000010)14.11.2018 22:27:53 - test (127.0.0.1)> 450 TLS session of data connection has not resumed or the session does not match the control connection
(000010)14.11.2018 22:27:55 - test (127.0.0.1)> disconnected.

Steps to reproduce:

For completeness...

• Install FileZilla server (https://filezilla-project.org/)
• Add user "test", PW "test" and any shared folder in FileZilla
• Create xunit project in VS mit FluentFTP nuget package
• Add test from this issue
• open bash (or install ubuntu from windows store)
• install dotnet sdk in bash (https://www.microsoft.com/net/download)
• cd /mnt/c/... wherevery your vs project is
• dotnet test

[avonheimburg]

This is probably the same problem as #311. The solution suggested there, to disable TLS session resumption on the server side, is not really an option because it would make data connections hijackable by an attacker. The only secure way to do passive FTPS is using TLS session resumption. ProFTPD and VSFTPD default to using it, too.

[robinrodricks]

TLS session resumption? How hard is that to implement?

[cluetjen]

I think you should try to get in contact with the Microsoft team. I'm not the TLS specialist but it seems that it's not possible because the underlaying SslStream does not expose an api that would allow it.

[jmezach]

Any updates on this issue? Is this a problem with SslStream in .NET Core that this library can't workaround? Or is there something else we can try? We're currently being hit by this issue after moving our code from Windows to Linux.

[cluetjen]

Is this a problem with SslStream in .NET Core that this library can't workaround?

It seems to be an SslStream issue. There seems to be simply no api available that allows to reuse an existing session and on windows this is done "magically". (s.a. https://github.com/dotnet/corefx/issues/33506)

[robinrodricks]

If you can suggest a fix it's possible. We can't fix something impossible.

On Wed, Feb 27, 2019, 9:09 PM Christoph Lütjen <notifications@github.com wrote:

It seems to be an SslStream issue. There seems to be simply no api available that allows to reuse an existing session and on windows this is done "magically". (s.a. dotnet/corefx#33506 https://github.com/dotnet/corefx/issues/33506)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/robinrodricks/FluentFTP/issues/347#issuecomment-467910896, or mute the thread https://github.com/notifications/unsubscribe-auth/AGe6Kt04hrqU9ennu9NDMkZuaW5u5_5kks5vRqakgaJpZM4Yeoyx .

[robinrodricks]

I'm closing this. If there is a way to solve this, I'd appreciate the know how. We can re-open this thread once a solution emerges.

[wfurt]

Is this now broken on all platforms @robinrodricks with the security updates or is this still only problem on Linux?

[krishKM]

Can confirm this has spread to Windows world too. Since we applied latest windows updates, we are no longer able to upload using TLS, Explicit . Same error about reusing connection.

Disabling Filezilla "reuse tls session" is not an option for us.

hoping to see fixes soon.

[datvm]

Is this now broken on all platforms @robinrodricks with the security updates or is this still only problem on Linux?

I am on Windows 11 and still have the same problem. .NET team claims it's Windows/Linux issues but at this phase I think it's .NET issue. I understand that FluentFTP team probably can do nothing about this unless they implement SSL on their own.

[i3v]

@robinrodricks,

I'm closing this. If there is a way to solve this, I'd appreciate the know how. We can re-open this thread once a solution emerges.

The FTPS-Connection-using-GnuTLS seem to be the official fix now, isn't it?

[FanDjango]

@i3v

The FTPS-Connection-using-GnuTLS seem to be the official fix now, isn't it?

No. The "official" fix is for Microsoft to fix this. Win 11 is therefore the fix, in the end.

FluentFTP.GnuTLS is yet another attempt to mitigate the problem (for those find this acceptable) by providing a TSL Session Resume capable stream.

Others will need to accept that this is unfixable without changing the base premises.

It's not FluentFTP failing to correctly reuse the TLS session, it is the underlying MS SslStream.

MS fixed it but "requests" you to migrate to WIN 11 / .NET 8 (unsure) to enjoy the fix.

[FanDjango]

A previous user said:

I am on Windows 11 and still have the same problem.

Need more proof of this. Default protocol settings disable TLS 1.3, thus to really succeed you need to change the default to include TLS 1.3, we disabled it to avoid the endless issues caused by the abortive attempts to use TLS 1.3 (with session resume failure).

[i3v]

@FanDjango ,

Win 11 is therefore the fix, in the end.

Actually, I'm currently more interested in Android, PhilippC/keepass2android/issues/1617 in particular.

The FluentFTP.GnuTLS do have some known limitations, but it should work on any OS, regardless of that Microsoft fix, shouldn't it? (Well... OK, I do agree it's probably more a workaround than the real fix.)

MS fixed it but "requests" you to migrate to WIN 11 / .NET 8 (unsure) to enjoy the fix.

AFAIU from dotnet/runtime/issues/22977 and dotnet/runtime/pull/64369, they did something in 7.0 (I'm really not sure what exactly). This blog post also advertises "TLS Resume" getting fixed for Linux in dotNET 7.0. AFAIU, they say that they made the necessary changes for it to work automatically on Linux, just like it works on Windows (note that Windows 11 is not mentioned, so I'd assume they mean any Windows).

I'm really not an expert in this, but from dotnet/runtime/issues/85113 + dotnet/runtime/issues/84136 it looks like:

• there is a stateful (aka "Session Resumption using SessionID") variant that works on Linux and Windows (but not MacOS) in some newer .NET Runtime version. It is not directly related to TLS1.3 and/or Windows11.
• there is a stateless (aka "Session Ticket Resumption") variant that is part of the TLS1.3 and thus requires Windows 11 (not supported on Linux and MacOS).

So... They say it should just work with dotNET7.0 , even without FluentFTP.GnuTLS... Interesting. Am I missing something, is there any evidence that Windows 11 fixes this?

[FanDjango]

@i3v

Of course things might have progressed since we introduced FluentFTP.GnuTLS as a possible alternative for some use cases.

To be honest, I myself have not been following the developments, as things concerning TLS 1.3 and session resume have quitened down somewhat, so I am not totally up-to-date on this.