robinrodricks / FluentFTP

An FTP and FTPS client for .NET & .NET Standard, optimized for speed. Provides extensive FTP commands, File uploads/downloads, SSL/TLS connections, Automatic directory listing parsing, File hashing/checksums, File permissions/CHMOD, FTP proxies, FXP support, UTF-8 support, Async/await support, Powershell support and more. Written entirely in C#.
MIT License
3.14k stars 657 forks source link

Downloading two files with the same name #1633

Closed zagadeka closed 2 months ago

zagadeka commented 3 months ago

FTP Server OS: Unix

FTP Server Type: Pure-FTPd

Client Computer OS: Windows

FluentFTP Version: 51.0.0

Framework: .NET 8

I get an exception every time I try to download two files with the same name in different directories:

            await asyncFtpClient.GetFiles(new string[] { "20240807.log", "log\\20240807.log" });

. In my business case, I don't control the names, so I can't avoid this.

Logs :

>         Connect(False)
Response: 220 You will be disconnected after 15 minutes of inactivity. [739113,484d]
Status:   Detected FTP server: PureFTPd
Command:  USER ***
Response: 331 User *** OK. Password required [3ms]
Command:  PASS ***
Response: 230 OK. Current directory is / [6ms]
Command:  FEAT
Response: 211 End. [2ms]
Text encoding: System.Text.UTF8Encoding+UTF8EncodingSealed
Command:  OPTS UTF8 ON
Response: 200 OK, UTF-8 enabled [1ms]
Command:  SYST
Response: 215 UNIX Type: L8 [2ms]
Command:  PWD
Response: 257 "/" is your current location [1ms]
>         GetListing(null, Auto)
Command:  TYPE I
Response: 200 TYPE is now 8-bit binary [2ms]
>         OpenDataStreamAsync("MLSD /", 0)
>         OpenPassiveDataStreamAsync(AutoPassive, "MLSD /", 0)
Command:  EPSV
Response: 229 Extended Passive mode OK (|||30001|) [2ms]
Command:  MLSD /
Response: 150 Accepted data connection [2ms]
>         CloseDataStream()
Response: 226 7 matches total [12ms]
>         DownloadFiles("c:\deviceftpclient\download\a", System.String[], Overwrite, None)
       System.ArgumentException : An item with the same key has already been added. Key: 

c:\deviceftpclient\download\a\20240807.log
       Stack Trace:
            at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
            at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
            at FluentFTP.Client.BaseClient.BaseFtpClient.RecordFileToDownload(List`1 rules, List`1 results, Dictionary`2 shouldExist, List`1 toDownload, FtpListItem remoteFile, String localFile, String remoteFilePath)
            at FluentFTP.Client.BaseClient.BaseFtpClient.GetFilesToDownload2(String localFolder, IEnumerable`1 remotePaths, List`1 rules, List`1 results, Dictionary`2 shouldExist)
            at FluentFTP.AsyncFtpClient.DownloadFiles(String localDir, IEnumerable`1 remotePaths, FtpLocalExists existsMode, FtpVerify verifyOptions, FtpError errorHandling, CancellationToken token, IProgress`1 progress, List`1 rules)
FanDjango commented 3 months ago

In my business case, I don't control the names, so I can't avoid this

Ok. Have you looked at the src code to see if there is an obvious change that would fix this? It seems obvious what is happening. Perhaps you would want to contribute? Especially as it seems to detriment your business?

You do have a number of ways to circumvent this yourself, if you are inventive. Don't put files with the same filename part of the path into the same GetFiles API operation, do those in a second (or third or....). You could check this yourself.

Still, I consider this to be a bug to be fixed.

zagadeka commented 3 months ago

I realised that I do not expect a different way of working. This seems to be in line with the way the FTP listen command works.

FanDjango commented 3 months ago

I realised that I do not expect a different way of working. This seems to be in line with the way the FTP listen command works.

Not sure what you mean by that. I will reopen this. As soon as someone has time, depending on proirities, it is sure to be looked at in more detail and will be investigated.

zagadeka commented 3 months ago

Thank you for your reply. When writing the previous message I did not notice your reply. I now search the subdirectories and download files from one directory at a time. This way there are no duplicates.

FanDjango commented 2 months ago

Ok, I have analysed the code and the behaviour.

The description of the DownloadFiles API is perhaps a little misleading.

        /// <summary>
        /// Downloads the specified files into a local single directory.
        /// High-level API that takes care of various edge cases internally.
        /// Supports very large files since it downloads data in chunks.
        /// Same speed as <see cref="o:DownloadFile"/>.
        /// </summary>

Note the line: Downloads the specified files into a local single directory.

Repeat: A single local directory

So, if you are downloading, say { "test.log", "log\test.log" } into "C:\destinationdir" he will try to put "test.log" as "test.log" there, and then try to put "log\test.log" there, also as "test.log".

There is no proviso for recursion or for "diving down into subdirectories"

FanDjango commented 2 months ago

Anyway, this is worth closing as is.