search

robinrodricks / FluentFTP

An FTP and FTPS client for .NET & .NET Standard, optimized for speed. Provides extensive FTP commands, File uploads/downloads, SSL/TLS connections, Automatic directory listing parsing, File hashing/checksums, File permissions/CHMOD, FTP proxies, FXP support, UTF-8 support, Async/await support, Powershell support and more. Written entirely in C#.
MIT License
3.14k stars 657 forks source link

Date modified incorrect after download with overwrite from zOS #1640

Closed MihaIvan closed 2 months ago

MihaIvan commented 2 months ago

FTP Server OS: IBM zOS

FTP Server Type: IBM CS FTP (z/OS, OS/400)

Client Computer OS: Windows 11

FluentFTP Version: 51.0.0.0

Framework: .NET 8

The file I downloaded was created and modified on 8/24/2024. The Date Modified for the file being replaced was 8/14/2024. After downloading the file, the Date Modified is 5/15/2000.

Logs : The logs below are from logging to the console. I was writing to the console as well, so there are extra messages.


# Connect(False)
Status:   FluentFTP 51.0.0.0(.NET 6.0) FtpClient
Status:   Connecting(sync) FtpClient.FtpSocketStream(control) IP #1 = ***:21
Status:   Waiting for a response
Response: 220-FTPSERVE IBM FTP CS V2R4 at S0W1.DAL-EBIS.IHOST.COM, 21:16:54 on 2024-08-30.
Response: 220 Connection will close if idle for more than 5 minutes. [739127.887d]
Status:   Detected FTP server: IBMzOSFTP
Command:  USER ***
Status:   Waiting for response to: USER ***
Response: 331 Send password please. [51ms]
Command:  PASS ***
Status:   Waiting for response to: PASS ***
Response: 230 CMPF is logged on.  Working directory is "CMPF.". [467ms]
Command:  FEAT
Status:   Waiting for response to: FEAT
Response: 211 no Extensions supported [49ms]
Status:   Text encoding: System.Text.ASCIIEncoding+ASCIIEncodingSealed
Command:  SYST
Status:   Waiting for response to: SYST
Response: 215 MVS is the operating system of this server. FTP Server is running on z/OS. [50ms]
Status:   Active ServerHandler is: IBMzOSFTP
Status:   Listing parser set to: IBMzOS
Command:  SITE DATASETMODE
Status:   Waiting for response to: SITE DATASETMODE
Response: 200 SITE command was accepted [62ms]
Command:  SITE QUOTESOVERRIDE
Status:   Waiting for response to: SITE QUOTESOVERRIDE
Response: 200 SITE command was accepted [51ms]
Command:  SITE LISTLEVEL=0
Status:   Waiting for response to: SITE LISTLEVEL=0
Response: 200 SITE command was accepted [54ms]
Command:  SITE LISTLEVEL=2
Status:   Waiting for response to: SITE LISTLEVEL=2
Response: 200 SITE command was accepted [49ms]
Command:  PWD
Status:   Waiting for response to: PWD
Response: 257 "'CMPF.'" is working directory. [61ms]
Downloading 'cmp.bkup.syss.xfer' / J:\MF Backups\SystemControls.bkp

# DownloadFile("J:\MF Backups\SystemControls.bkp", "'cmp.bkup.syss.xfer'", Overwrite, None)

# OpenRead("'cmp.bkup.syss.xfer'", Binary, 0, 0, False)

# GetFileSize("'cmp.bkup.syss.xfer'")

# GetListing("'cmp.bkup.syss.xfer'", Auto)
Command:  TYPE I
Status:   Waiting for response to: TYPE I
Response: 200 Representation type is Image [59ms]

# OpenDataStream("LIST 'cmp.bkup.syss.xfer'", 0)

# OpenPassiveDataStream(AutoPassive, "LIST 'cmp.bkup.syss.xfer'", 0)
Command:  EPSV
Status:   Waiting for response to: EPSV
Response: 229 Entering Extended Passive Mode (|||1063|) [50ms]
Status:   Connecting(sync) FtpClient.FtpSocketStream(data) IP #1 = ***:1063
Command:  LIST 'cmp.bkup.syss.xfer'
Status:   Waiting for response to: LIST 'cmp.bkup.syss.xfer'
Response: 125 List started OK [320ms]
+---------------------------------------+
Listing:  Volume Referred      Ext      Used Recfm Lrecl BlkSz Dsorg Dsname
Listing:  CMP029 2024/08/30      1        30  FB    1024 27648  PS   'CMP.BKUP.SYSS.XFER'
-----------------------------------------
Status:   Disposing(sync) FtpClient.FtpSocketStream(data)

# CloseDataStream()
Status:   Waiting for response to: LIST 'cmp.bkup.syss.xfer'
Response: 250 List completed successfully. [377ms]
Status:   Disposing(sync) FtpClient.FtpSocketStream(data) (redundant)
Status:   Confirmed format IBMzOS

# OpenDataStream("RETR 'cmp.bkup.syss.xfer'", 0)

# OpenPassiveDataStream(AutoPassive, "RETR 'cmp.bkup.syss.xfer'", 0)
Command:  EPSV
Status:   Waiting for response to: EPSV
Response: 229 Entering Extended Passive Mode (|||1064|) [50ms]
Status:   Connecting(sync) FtpClient.FtpSocketStream(data) IP #1 = ***:1064
Command:  RETR 'cmp.bkup.syss.xfer'
Status:   Waiting for response to: RETR 'cmp.bkup.syss.xfer'
Response: 125 Sending data set CMP.BKUP.SYSS.XFER FIXrecfm 1024 [299ms]
Status:   Downloaded 1199104 bytes, 1.178s, 993.2 KB/s
Status:   Disposing(sync) FtpClient.FtpSocketStream(data)
Status:   Waiting for response to: RETR 'cmp.bkup.syss.xfer'
Response: 250 Transfer completed successfully. [1.605s]
Downloaded : 'cmp.bkup.syss.xfer' / J:\MF Backups\SystemControls.bkp
    Elapsed time: 2.287s
File Downloaded complete
FanDjango commented 2 months ago

So: a file from the remote server is downloaded to your local client.

If a LIST command is issued to the server, the output will be parsed and things like size, date etc. will be populated into a returned data structure. This parsing is done in a fashion specific to the servers quirks. It might or might not contain all information that other servers provide.

But generally, for a file download/upload, FluentFTP currently does not attempt to acquire the remote/local files meta data (creation date, modified date, permissions, ownership etc. etc.) and thus also does not attempt to set these after writing the file locally/remotely.

This behaviour is not specific to z/OS. It is thus for all server types.

FanDjango commented 2 months ago

The only strange thing you might like to investigate, is

After downloading the file, the Date Modified is 5/15/2000

FluentFTP certainly did not set that.

MihaIvan commented 2 months ago

OK, this is weird. That last Win 11 update really messed things up. First, I had to abandon the native Win FTP because it killed being able to transfer files. Operating in Passive mode is not supported, so it just fails. I have been transferring to an external share drive. Up until this last release, the date modified was working fine. Now when files are put on the drive, the all the dates get corrupted. Local drive is fine.

So I think we can close this problem because it is a Windows problem not yours. Thanks.

FanDjango commented 2 months ago

I will be happy if you report your further results investigating this. But note: FTP itself does not pass this meta file information - and "primitive" clients do not try to replicate these. One could try to implement this, and FluentFTP would actually give you the means to do this. But consider the needed logic, especially if you go beyond dates for creation, modification and last read-access, to replicate ACLs and permissions etc. It was never the purpose if FTP initially, although some extensions tried to go in that direction.