robinrodricks / FluentFTP

An FTP and FTPS client for .NET & .NET Standard, optimized for speed. Provides extensive FTP commands, File uploads/downloads, SSL/TLS connections, Automatic directory listing parsing, File hashing/checksums, File permissions/CHMOD, FTP proxies, FXP support, UTF-8 support, Async/await support, Powershell support and more. Written entirely in C#.
MIT License
3.08k stars 650 forks source link

FtpResult.Size is zero #1639

Closed zagadeka closed 3 weeks ago

zagadeka commented 1 month ago

FTP Server OS: Unix

FTP Server Type: Pure-FTPd

Client Computer OS: Windows

FluentFTP Version: 51.0.0

Framework: .NET 8

All results of the DownloadFiles method of the AsyncFtpClient client have the Size property set to zero despite outputting the correct file size in the log.

Logs :


[2024-08-28 08:22:15 INF] >         Connect(False)
[2024-08-28 08:22:15 DEB] FluentFTP 51.0.0.0(.NET 6.0) AsyncFtpClient
[2024-08-28 08:22:15 DEB] Connecting(async) AsyncFtpClient.FtpSocketStream(control) IP #1 = ***:21
[2024-08-28 08:22:16 DEB] Waiting for a response
[2024-08-28 08:22:17 DEB] Response: 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
Response: 220-You are user number 2 of 5 allowed.
Response: 220-Local time is now 08:22. Server port: 21.
Response: 220-This is a private system - No anonymous login
Response: 220-IPv6 connections are also welcome on this server.
[2024-08-28 08:22:17 INF] Response: 220 You will be disconnected after 15 minutes of inactivity. [739125,349d]
[2024-08-28 08:22:17 INF] Status:   Detected FTP server: PureFTPd
[2024-08-28 08:22:17 INF] Command:  USER ***
[2024-08-28 08:22:17 DEB] Waiting for response to: USER ***
[2024-08-28 08:22:17 INF] Response: 331 User *** OK. Password required [3ms]
[2024-08-28 08:22:17 INF] Command:  PASS ***
[2024-08-28 08:22:17 DEB] Waiting for response to: PASS ***
[2024-08-28 08:22:17 INF] Response: 230 OK. Current directory is / [13ms]
[2024-08-28 08:22:17 INF] Command:  FEAT
[2024-08-28 08:22:17 DEB] Waiting for response to: FEAT
[2024-08-28 08:22:17 DEB] Response: 211-Extensions supported:
Response: EPRT
Response: IDLE
Response: MDTM
Response: SIZE
Response: MFMT
Response: REST STREAM
Response: MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
Response: MLSD
Response: AUTH TLS
Response: PBSZ
Response: PROT
Response: UTF8
Response: TVFS
Response: ESTA
Response: PASV
Response: EPSV
Response: SPSV
[2024-08-28 08:22:17 INF] Response: 211 End. [12ms]
[2024-08-28 08:22:17 INF] Text encoding: System.Text.UTF8Encoding+UTF8EncodingSealed
[2024-08-28 08:22:17 INF] Command:  OPTS UTF8 ON
[2024-08-28 08:22:17 DEB] Waiting for response to: OPTS UTF8 ON
[2024-08-28 08:22:17 INF] Response: 200 OK, UTF-8 enabled [3ms]
[2024-08-28 08:22:17 INF] Command:  SYST
[2024-08-28 08:22:17 DEB] Waiting for response to: SYST
[2024-08-28 08:22:17 INF] Response: 215 UNIX Type: L8 [5ms]
[2024-08-28 08:22:17 DEB] Active ServerHandler is: PureFTPd
[2024-08-28 08:22:17 DEB] Listing parser set to: Machine
[2024-08-28 08:22:17 INF] Command:  PWD
[2024-08-28 08:22:17 DEB] Waiting for response to: PWD
[2024-08-28 08:22:17 INF] Response: 257 "/" is your current location [3ms]
[2024-08-28 08:22:17 INF] >         GetListing("", Auto)
[2024-08-28 08:22:17 INF] Command:  TYPE I
[2024-08-28 08:22:17 DEB] Waiting for response to: TYPE I
[2024-08-28 08:22:17 INF] Response: 200 TYPE is now 8-bit binary [4ms]
[2024-08-28 08:22:17 INF] >         OpenDataStreamAsync("MLSD /", 0)
[2024-08-28 08:22:17 INF] >         OpenPassiveDataStreamAsync(AutoPassive, "MLSD /", 0)
[2024-08-28 08:22:17 INF] Command:  EPSV
[2024-08-28 08:22:17 DEB] Waiting for response to: EPSV
[2024-08-28 08:22:17 INF] Response: 229 Extended Passive mode OK (|||30004|) [4ms]
[2024-08-28 08:22:17 DEB] Connecting(async) AsyncFtpClient.FtpSocketStream(data) IP #1 = ***:30004
[2024-08-28 08:22:18 INF] Command:  MLSD /
[2024-08-28 08:22:18 DEB] Waiting for response to: MLSD /
[2024-08-28 08:22:18 INF] Response: 150 Accepted data connection [4ms]
[2024-08-28 08:22:18 DEB] +---------------------------------------+
[2024-08-28 08:22:18 DEB] Listing:  type=cdir;sizd=4096;modify=20240823062744;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d00; .
[2024-08-28 08:22:18 DEB] Listing:  type=pdir;sizd=4096;modify=20240823062744;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d00; ..
[2024-08-28 08:22:18 DEB] Listing:  type=file;size=12287439;modify=20240808085758;UNIX.mode=0644;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d04; 20240806.lg2
[2024-08-28 08:22:18 DEB] Listing:  type=file;size=24575869;modify=20240808090148;UNIX.mode=0644;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d05; 20240807.lg2
[2024-08-28 08:22:18 DEB] Listing:  type=file;size=12287439;modify=20240808085758;UNIX.mode=0644;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d06; 20240808.12.log
[2024-08-28 08:22:18 DEB] Listing:  type=dir;sizd=4096;modify=20240817075410;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1cf7; log
[2024-08-28 08:22:18 DEB] Listing:  type=dir;sizd=4096;modify=20240823062759;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d02; log.old
[2024-08-28 08:22:18 DEB] -----------------------------------------
[2024-08-28 08:22:18 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(data)
[2024-08-28 08:22:18 INF] >         CloseDataStream()
[2024-08-28 08:22:18 DEB] Waiting for response to: MLSD /
[2024-08-28 08:22:18 DEB] Response: 226-Options: -a -l
[2024-08-28 08:22:18 INF] Response: 226 7 matches total [29ms]
[2024-08-28 08:22:18 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(data) (redundant)
[2024-08-28 08:22:18 INF] >         GetListing("/log", Auto)
[2024-08-28 08:22:18 INF] >         OpenDataStreamAsync("MLSD /log", 0)
[2024-08-28 08:22:18 INF] >         OpenPassiveDataStreamAsync(AutoPassive, "MLSD /log", 0)
[2024-08-28 08:22:18 INF] Command:  EPSV
[2024-08-28 08:22:18 DEB] Waiting for response to: EPSV
[2024-08-28 08:22:18 INF] Response: 229 Extended Passive mode OK (|||30001|) [3ms]
[2024-08-28 08:22:18 DEB] Connecting(async) AsyncFtpClient.FtpSocketStream(data) IP #1 = ***:30001
[2024-08-28 08:22:19 INF] Command:  MLSD /log
[2024-08-28 08:22:19 DEB] Waiting for response to: MLSD /log
[2024-08-28 08:22:19 INF] Response: 150 Accepted data connection [5ms]
[2024-08-28 08:22:19 DEB] +---------------------------------------+
[2024-08-28 08:22:19 DEB] Listing:  type=cdir;sizd=4096;modify=20240817075410;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1cf7; .
[2024-08-28 08:22:19 DEB] Listing:  type=pdir;sizd=4096;modify=20240823062744;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d00; ..
[2024-08-28 08:22:19 DEB] Listing:  type=file;size=12287439;modify=20240808085758;UNIX.mode=0644;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d01; 20240807.log
[2024-08-28 08:22:19 DEB] -----------------------------------------
[2024-08-28 08:22:19 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(data)
[2024-08-28 08:22:19 INF] >         CloseDataStream()
[2024-08-28 08:22:19 DEB] Waiting for response to: MLSD /log
[2024-08-28 08:22:19 DEB] Response: 226-Options: -a -l
[2024-08-28 08:22:19 INF] Response: 226 3 matches total [25ms]
[2024-08-28 08:22:19 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(data) (redundant)
[2024-08-28 08:22:19 INF] >         GetListing("/log.old", Auto)
[2024-08-28 08:22:19 INF] >         OpenDataStreamAsync("MLSD /log.old", 0)
[2024-08-28 08:22:19 INF] >         OpenPassiveDataStreamAsync(AutoPassive, "MLSD /log.old", 0)
[2024-08-28 08:22:19 INF] Command:  EPSV
[2024-08-28 08:22:19 DEB] Waiting for response to: EPSV
[2024-08-28 08:22:19 INF] Response: 229 Extended Passive mode OK (|||30005|) [4ms]
[2024-08-28 08:22:19 DEB] Connecting(async) AsyncFtpClient.FtpSocketStream(data) IP #1 = ***:30005
[2024-08-28 08:22:20 INF] Command:  MLSD /log.old
[2024-08-28 08:22:20 DEB] Waiting for response to: MLSD /log.old
[2024-08-28 08:22:20 INF] Response: 150 Accepted data connection [6ms]
[2024-08-28 08:22:20 DEB] +---------------------------------------+
[2024-08-28 08:22:20 DEB] Listing:  type=cdir;sizd=4096;modify=20240823062759;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d02; .
[2024-08-28 08:22:20 DEB] Listing:  type=pdir;sizd=4096;modify=20240823062744;UNIX.mode=0755;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d00; ..
[2024-08-28 08:22:20 DEB] Listing:  type=file;size=12287439;modify=20240808085758;UNIX.mode=0644;UNIX.uid=8;UNIX.gid=401;unique=92g2fc1d03; 20240805lg2
[2024-08-28 08:22:20 DEB] -----------------------------------------
[2024-08-28 08:22:20 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(data)
[2024-08-28 08:22:20 INF] >         CloseDataStream()
[2024-08-28 08:22:20 DEB] Waiting for response to: MLSD /log.old
[2024-08-28 08:22:20 DEB] Response: 226-Options: -a -l
[2024-08-28 08:22:20 INF] Response: 226 3 matches total [25ms]
[2024-08-28 08:22:20 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(data) (redundant)
[2024-08-28 08:22:20 INF] Download files /20240808.12.log from the device 2097153 to the directory download\23bc1b33-e95f-413a-80af-db4a5fa4e16e started
[2024-08-28 08:22:20 INF] >         DownloadFiles("download\23bc1b33-e95f-413a-80af-db4a5fa4e16e", System.Linq.Enumerable+WhereEnumerableIterator`1[System.String], Skip, Retry, Delete)
[2024-08-28 08:22:20 INF] >         DownloadFile("download\23bc1b33-e95f-413a-80af-db4a5fa4e16e\20240808.12.log", "/20240808.12.log", Skip, Retry, Delete)
[2024-08-28 08:22:20 INF] >         OpenRead("/20240808.12.log", Binary, 0, 0, False)
[2024-08-28 08:22:20 INF] >         GetFileSize("/20240808.12.log", -1)
[2024-08-28 08:22:20 INF] Command:  SIZE /20240808.12.log
[2024-08-28 08:22:20 DEB] Waiting for response to: SIZE /20240808.12.log
[2024-08-28 08:22:20 INF] Response: 213 12287439 [4ms]
[2024-08-28 08:22:20 INF] >         OpenDataStreamAsync("RETR /20240808.12.log", 0)
[2024-08-28 08:22:20 INF] >         OpenPassiveDataStreamAsync(AutoPassive, "RETR /20240808.12.log", 0)
[2024-08-28 08:22:20 INF] Command:  EPSV
[2024-08-28 08:22:20 DEB] Waiting for response to: EPSV
[2024-08-28 08:22:20 INF] Response: 229 Extended Passive mode OK (|||30006|) [5ms]
[2024-08-28 08:22:20 DEB] Connecting(async) AsyncFtpClient.FtpSocketStream(data) IP #1 = ***:30006
[2024-08-28 08:22:21 INF] Command:  RETR /20240808.12.log
[2024-08-28 08:22:21 DEB] Waiting for response to: RETR /20240808.12.log
[2024-08-28 08:22:21 DEB] Response: 150-Accepted data connection
[2024-08-28 08:22:21 INF] Response: 150 11999.5 kbytes to download [8ms]
[2024-08-28 08:22:22 DEB] Downloaded 12287439 bytes, 513ms, 22,8 MB/s
[2024-08-28 08:22:22 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(data)
[2024-08-28 08:22:22 DEB] Waiting for response to: RETR /20240808.12.log
[2024-08-28 08:22:22 DEB] Response: 226-File successfully transferred
[2024-08-28 08:22:22 INF] Response: 226 0.146 seconds (measured here), 80.47 Mbytes per second [537ms]
[2024-08-28 08:22:22 INF] Source server does not support any common hashing algorithm
[2024-08-28 08:22:22 INF] Falling back to file size comparison
[2024-08-28 08:22:22 INF] >         GetFileSize("/20240808.12.log", -1)
[2024-08-28 08:22:22 INF] Command:  SIZE /20240808.12.log
[2024-08-28 08:22:22 DEB] Waiting for response to: SIZE /20240808.12.log
[2024-08-28 08:22:22 INF] Response: 213 12287439 [4ms]
[2024-08-28 08:22:22 INF] File Verification: PASS
[2024-08-28 08:22:22 INF] >         Disconnect()
[2024-08-28 08:22:22 INF] Command:  QUIT
[2024-08-28 08:22:22 DEB] Waiting for response to: QUIT
[2024-08-28 08:22:22 DEB] Response: 221-Goodbye. You uploaded 0 and downloaded 12000 kbytes.
[2024-08-28 08:22:22 INF] Response: 221 Logout. [4ms]
[2024-08-28 08:22:22 DEB] Disposing(async) AsyncFtpClient.FtpSocketStream(control)
[2024-08-28 08:22:22 INF] >         DisposeAsync()
[2024-08-28 08:22:22 DEB] Disposing(async) AsyncFtpClient
[2024-08-28 08:22:22 INF] >         Disconnect()
[2024-08-28 08:22:22 DEB] Connection already closed, nothing to do.
FanDjango commented 4 weeks ago

As soon as my schedule allows, I will look into this.

FanDjango commented 4 weeks ago

@robinrodricks Here you have a server that responds (FEAT) with

Response: MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
Response: MLSD

by the way (non RFC compliant) and probably it is telling us it supports both MLST andMLSD** (which is redundant, as MLST implies MLSD).

For this issue, it needs to be checked if the MLST format ("type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique") is standard and parsed correctly by FluentFTP before checking anything else, I suppose.

FanDjango commented 3 weeks ago

Ok, I can confirm this happens both with SYNC and ASYNC.

FanDjango commented 3 weeks ago

Well, I can see in the code that the only information actually returned in the FtpResult list is the FtpStatus, (i.e. skipped, failed or success). And actually the XML doc for DownloadFiles states this:

        /// <returns>
        /// Returns a listing of all the remote files, indicating if they were downloaded, skipped or overwritten.
        /// Returns a blank list if nothing was transferred. Never returns null.
        /// </returns>

So, you have the local and the remote paths and only the FtpStatus. No Size.

I suppose you would like to have the size at that point, but it is not returned down the call stack from the single file download. Changing that would entail a lot of work.

You could get the size yourself if you want it, either locally or remote, after the transfer. If you need it for verification, note that the file download(s) can be optionally set to verify by size, hash or whatever.

zagadeka commented 3 weeks ago

@FanDjango: Thank you for your reply. In that case, the DownloadFiles method should not return the FtpStatus collection as this is confusing.

I will ask about the size independently. I suppose that in general the size in the operation may differ from the size of the original file, especially in the case of failure.