lavv17 / lftp

sophisticated command line file transfer program (ftp, http, sftp, fish, torrent)
http://lftp.yar.ru
GNU General Public License v3.0
1.08k stars 159 forks source link

TLS1.3 - FTP data truncated #666

Closed AnaelMobilia closed 2 years ago

AnaelMobilia commented 2 years ago

Hello,

I'm facing an issue with TLS1.3: data are truncated when sending files through FTPS connection (FTP with a TLS encryption).

My environment :

When restricting OpenSSL on the server to TLS1.2 (/etc/ssl/openssl.cnf -> MaxProtocol = TLS1.2), there is no issue when sending the file through lftp (using Ubuntu or Debian).

When using TLS1.3, there is an issue with all OS except Ubuntu 22.04: data are truncated => a 101701 bytes file is only transferred for 65536 bytes, the end of the file isn't transmitted. On (lftp) debug mode, I get an error gnutls_record_recv: The TLS connection was non-properly terminated. Assuming EOF.

I notice that the 65536 bytes looks like 4 packets with a payload of 16384 bytes (max payload for a TLS packet).

See the matrix below for the results of my tests :

OS OpenSSL version lftp version GnuTLS version TLS1.2 TLS1.3
Ubuntu 20.04 LTS 1.1.1f 31 Mar 2020 4.8.4 3.6.13 Success Failure
Ubuntu 22.04 LTS 3.0.2 15 Mar 2022 4.9.2 3.7.3 Success Success
Debian 10 1.1.1n 15 Mar 2022 4.8.4 3.6.7 Success Failure
Debian 11 1.1.1n 15 Mar 2022 4.8.4 3.7.1 Success Failure
Debian testing 1.1.1o 3 May 2022 4.9.2 3.7.4 Success Failure

Nothing specific on server log (pure-ftpd) :

May  9 16:07:57 vm pure-ftpd: (?@172.16.10.3) [INFO] New connection from 172.16.10.3
May  9 16:07:57 vm pure-ftpd: (?@172.16.10.3) [DEBUG] Command [feat] []
May  9 16:07:57 vm pure-ftpd: (?@172.16.10.3) [DEBUG] Command [auth] [TLS]
May  9 16:07:58 vm pure-ftpd: (?@172.16.10.3) [INFO] TLS: Enabled TLSv1.3 with TLS_AES_256_GCM_SHA384, 256 secret bits cipher
May  9 16:07:58 vm pure-ftpd: (?@172.16.10.3) [DEBUG] Command [opts] [UTF8 ON]
May  9 16:07:58 vm pure-ftpd: (?@172.16.10.3) [DEBUG] Command [user] [<username>]
May  9 16:07:58 vm pure-ftpd: (?@172.16.10.3) [DEBUG] Command [pass] [<*>]
May  9 16:07:58 vm pure-ftpd: (?@172.16.10.3) [INFO] <username> is now logged in
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [opts] [MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid;]
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [pwd] []
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [pbsz] [0]
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [prot] [P]
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [type] [I]
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [pasv] []
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [stor] [file.csv]
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [INFO] TLS: Enabled TLSv1.3 with TLS_AES_256_GCM_SHA384, 256 secret bits cipher
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [NOTICE] /data/file.csv uploaded  (65536 bytes, 1204.34KB/sec)
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [mfmt] [20220505142652 file.csv]
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [DEBUG] Command [quit] []
May  9 16:07:58 vm pure-ftpd: (<username>@172.16.10.3) [INFO] Logout.

I tested a connection through the following TLS1.3 ciphers:

The result is the same for each ciphers : data are truncated.

Another interesting information : when putting lftp on debug mode (-d), sometimes (~20%) file transfer is successful. Looks like a little sleep between two TLS packets help (a buffer which need a little time to empty ?).

Bests regards, Anael