FTP Error

[cclague]

Describe the bug When trying to FTP results, it says it published successfully, but it didn't work.

I have recently moved my ftp server to Docker which I suspect is the cause of this issue - I have exposed ports 20 and 21. If I do Configure FTP / Test FTP Connection in CrossMgr I get a "FTP Test Successful" message. I am also able to use FTP from a command line client with no issues, but when I use CrossMgr no files are sent.

To Reproduce Steps to reproduce the behavior:

1. Publish results with Ftp selected for some/any files.

Expected behavior File(s) sent to FTP server

Desktop (please complete the following information):

• Windows 10 Home (10.0.18362)

Additional context Log file content:

2020-11-15T18:02:59 (v3.0.58-20201104092209 Windows) start: CrossMgr 3.0.58-20201104092209 2020-11-15T18:02:59 (v3.0.58-20201104092209 Windows) lang: "en" 2020-11-15T18:03:00 (v3.0.58-20201104092209 Windows) page: Actions 2020-11-15T18:03:00 (v3.0.58-20201104092209 Windows) page: Keypad 2020-11-15T18:03:01 (v3.0.58-20201104092209 Windows) page: GeneralInfoProperties 2020-11-15T18:03:16 (v3.0.58-20201104092209 Windows) CrossMgr 3.0.58-20201104092209: Windows 10 2020-11-15T18:03:16 (v3.0.58-20201104092209 Windows) call: openRace: "C:\Users\LCCA\CrossMgrRaces\LCCA Team Championships - Senior - Club\2019-02-10-LCCA Team Championships - Senior - Club-Club Team Champs-r1-.cmn" 2020-11-15T18:03:16 (v3.0.58-20201104092209 Windows) openRace: Excel file "C:\Users\LCCA\CrossMgrRaces\LCCA Team Championships - Senior - Club\2019-02-10-LCCA Team Championships - Senior - Club-Club Team Champs.xlsx" 2020-11-15T18:32:33 (v3.0.58-20201104092209 Windows) call: menuPublishBatch(<<MainWin>>, <<CommandEvent>>) 2020-11-15T18:32:44 (v3.0.58-20201104092209 Windows) call: menuPublishHtmlRaceResults(<<MainWin>>, silent=True) 2020-11-15T18:32:44 (v3.0.58-20201104092209 Windows) call: menuPublishHtmlIndex(<<MainWin>>, silent=True) 2020-11-15T18:32:44 (v3.0.58-20201104092209 Windows) call: menuPrintPDF(<<MainWin>>, silent=True) 2020-11-15T18:32:45 (v3.0.58-20201104092209 Windows) call: menuPublishAsExcel(<<MainWin>>, silent=True) 2020-11-15T18:33:07 (v3.0.58-20201104092209 Windows) FtpUploadFile:FTPOSError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond Debugging info: ftputil 4.0.0, Python 3.7.9 (win32)

[esitarski]

OK. The relevant entry in the log is:

2020-11-15T18:33:07 (v3.0.58-20201104092209 Windows) FtpUploadFile:FTPOSError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond Debugging info: ftputil 4.0.0, Python 3.7.9 (win32)

Which is saying that the ftp connection is timing out.

Port 21 is OK for ftp. Port 20 is not used.

I can't find p[roblems with the ftputil module I am using to communicate with ftp in docker. Perhaps there is something missing in the ftp configuration in CrossMgr that is different from using ftp on the command line. I just checked in the updates, so a new build should be available in about 20 minutes from now.

The best I can do is to write all the parameters to FtpWriteFile call into the log, not just the error.

Here is what you need to do? Download and install CrossMgr 3.0.59 Beta. Run it, and check the log. It will now include all parameters of the FtpWriteFile call. If any are not correct, fix up the ftp configuration and it should work.

If it doesn't send me the new log and we can go from there.

On Sun, Nov 15, 2020 at 1:49 PM cclague notifications@github.com wrote:

Describe the bug When trying to FTP results, it says it published successfully, but it didn't work.

I have recently moved my ftp server to Docker which I suspect is the cause of this issue - I have exposed ports 20 and 21. If I do Configure FTP / Test FTP Connection in CrossMgr I get a "FTP Test Successful" message. I am also able to use FTP from a command line client with no issues, but when I use CrossMgr no files are sent.

To Reproduce Steps to reproduce the behavior:

1. Publish results with Ftp selected for some/any files.

Expected behavior File(s) sent to FTP server

Desktop (please complete the following information):

• Windows 10 Home (10.0.18362)

Additional context Log file content:

2020-11-15T18:02:59 (v3.0.58-20201104092209 Windows) start: CrossMgr 3.0.58-20201104092209 2020-11-15T18:02:59 (v3.0.58-20201104092209 Windows) lang: "en" 2020-11-15T18:03:00 (v3.0.58-20201104092209 Windows) page: Actions 2020-11-15T18:03:00 (v3.0.58-20201104092209 Windows) page: Keypad 2020-11-15T18:03:01 (v3.0.58-20201104092209 Windows) page: GeneralInfoProperties 2020-11-15T18:03:16 (v3.0.58-20201104092209 Windows) CrossMgr 3.0.58-20201104092209: Windows 10 2020-11-15T18:03:16 (v3.0.58-20201104092209 Windows) call: openRace: "C:\Users\LCCA\CrossMgrRaces\LCCA Team Championships - Senior - Club\2019-02-10-LCCA Team Championships - Senior - Club-Club Team Champs-r1-.cmn" 2020-11-15T18:03:16 (v3.0.58-20201104092209 Windows) openRace: Excel file "C:\Users\LCCA\CrossMgrRaces\LCCA Team Championships - Senior - Club\2019-02-10-LCCA Team Championships - Senior - Club-Club Team Champs.xlsx" 2020-11-15T18:32:33 (v3.0.58-20201104092209 Windows) call: menuPublishBatch(<>, <>) 2020-11-15T18:32:44 (v3.0.58-20201104092209 Windows) call: menuPublishHtmlRaceResults(<>, silent=True) 2020-11-15T18:32:44 (v3.0.58-20201104092209 Windows) call: menuPublishHtmlIndex(<>, silent=True) 2020-11-15T18:32:44 (v3.0.58-20201104092209 Windows) call: menuPrintPDF(<>, silent=True) 2020-11-15T18:32:45 (v3.0.58-20201104092209 Windows) call: menuPublishAsExcel(<>, silent=True) 2020-11-15T18:33:07 (v3.0.58-20201104092209 Windows) FtpUploadFile:FTPOSError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond Debugging info: ftputil 4.0.0, Python 3.7.9 (win32)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/esitarski/CrossMgr/issues/69, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABGXKMECV3DBJXLCEIXXKDSQAPFNANCNFSM4TWLPOXA .

--

Edward Sitarski

[cclague]

Thanks - I think there's en error in that build - if I do an FTP test I get: FtpWriteFile() got an unexpected keyword argument 'hostname'

and the same in the log if I ignore that and attempt to run the upload. Possibly server not hostname?

[esitarski]

Apologies. A new update will be available in about min.

On Mon, Nov 16, 2020 at 1:37 PM cclague notifications@github.com wrote:

Thanks - I think there's en error in that build - if I do an FTP test I get: FtpWriteFile() got an unexpected keyword argument 'hostname'

and the same in the log if I ignore that and attempt to run the upload. Possibly server not hostname?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/esitarski/CrossMgr/issues/69#issuecomment-728248076, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABGXKIRNCZE3RBKOQWTZ3TSQFWMZANCNFSM4TWLPOXA .

--

Edward Sitarski

[cclague]

OK - I'm sure this must be a config issue on my side - I've had this working in previous versions of crossmgr. It is strange though - from the ftp server I can see crossmgr connecting and disconnecting but it's not sending the file - I guess it's waiting for some sort of handshake or something. The logs on the ftp server are empty so it's not even trying to send the file. I guess this is consistent with the message in the crossmgr log where it says it didn't get a response. Frustrating though.

[mbuckaway]

There used to be some weird issue related to a directory on the FTP server that either didn't exist or needed to exist to upload the file....or it didn't like using the root directory or something. It's a problem I found and fixed...which looks like it back. I suggest making sure files actually got generated locally in your race directory and are viewable from the CM web server.

[cclague]

hmm - there may be something in this. If I set a folder in 'Upload files to Path' in CrossMgr it will create that folder on the ftp server, but still won't upload the files to it.

In terms of the local files being viewable from the CM web server, that appears to work fine.

[esitarski]

Two things:

1: I just discovered that ftputils.makedirs has a new exist_ok parameter which I now set to True. This avoids swallowing an Exception and might help shed some light on the problem. Available in the latest release build.

2: While we puzzle this out, have you tried the "Post Publish Command" as a workaround? This is configured in the "Batch Publish" dialog. The characters *% expand to all the files CrossMgr wants to upload. The file names are properly quoted to work in the command line. If you enter your own ftp upload command, does it work? If so, what command are you using?

Thanks.

On Mon, Nov 16, 2020 at 6:06 PM cclague notifications@github.com wrote:

hmm - there may be something in this. If I set a folder in 'Upload files to Path' in CrossMgr it will create that folder on the ftp server, but still won't upload the files to it.

In terms of the local files being viewable from the CM web server, that appears to work fine.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/esitarski/CrossMgr/issues/69#issuecomment-728383582, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABGXKKNGLBYC5GZSCAXTQLSQGV7LANCNFSM4TWLPOXA .

--

Edward Sitarski

[cclague]

You're testing my dos skills now...

So to get a single line command script to run from the command prompt I created the following ftpupload.bat script: echo open 192.168.1.101 >> ftp &echo user ftpuser PASSWORD>> ftp &echo binary >> ftp &echo put %1 >> ftp &echo bye >> ftp &ftp -n -v -s:ftp &del ftp

running from the command line (being as explicit as possible here for clarity): c:\users\lcca\ftpload.bat c:\users\lcca\crossmgrlog.txt

This works correctly and transfers the log file.

Moving to CrossMgr, I'm inserting the following into the post publish command field and un selecting the ftp : c:\users\lcca\ftpload.bat %* Which says it was successful, but did nothing. There is nothing in the CrossMgr log either.

Moving on I'm trying the command directly: echo open 192.168.1.101 >> ftp &echo user ftpuser PASSWORD >> ftp &echo binary >> ftp &echo put %* >> ftp &echo bye >> ftp &ftp -n -v -s:ftp &del ftp

Perhaps unsurprisingly this has the same effect (i.e. nothing happens)

[cclague]

OK - I carried on digging some more... I created a new folder with a much shorter/simpler name and renamed the race (so now neither contained spaces). If I publish a single file (so this is down to my dos skills ;-) ) then now the post publish works, so there was something in the passing of the filename that was being truncated or manipulated I think. However the ftp does not work still.

Failed: fname="['C:\\Users\\LCCA\\CrossMgrRaces\\LCCA Team Championships - Senior - Club\\2019-02-10-LCCA Team Championships - Senior - Club-Club Team Champs-r1-.html']"

Succeeded: fname="['C:\\Users\\LCCA\\CrossMgrRaces\\tester\\2019-02-10-LCCAR1-r1-.xlsx']"

[esitarski]

Your "Failed" path has 148 characters - above 127 characters Your "Succeeded" file is 66 characters - below 127 characters.

This is suspicious.

There are a number of reports on the internet about FTP problems and a maximum file path of 127 characters (see here).

FTP has no problem with long path lengths, but EW2 and EW3 can be configured with a 127 char path maximum. In CrossMgr, all FTP failures are reported. However, since the problem isn't in FTP, it may not get an error. If FTP doesn't get an error, CrossMgr isn't going to report it.

My suggestion is that you try to FTP to this server with another application other than CrossMgr (cmd line, or app). If it also has a max 127 path limit, the problem is with how your server was configured.

After reviewing the source code, I can assure you that CrossMgr will faithfully report all FTP errors.