Open DrVanScott opened 6 years ago
I just edited the inital comment, added some test configurations.
I've now enabled debug logging in my test scenario on both sides. Please find it attached. I have shortened it to the relevant (but complete) range on both sides.
EDIT: The log of maverick would have also contained the transferred data. As this would slow down the whole process and also would made the log hard to read, i removed that. There is just a single line not handled by my filter by accident, but it is also indicating EOF has reached ;-)
INFO: Sent 8116 bytes of transport data outgoingSequence=16489 totalBytesSinceKEX=270141565
INFO: Session OUT: 29999105
30000000
INFO: Sending SSH_MSG_CHANNEL_DATA id=0 rid=0 len=1 window=106624
Hi,
I use this library to transfer files sized up to some gigabytes. Sometimes, quite rare, such a transfer doesn't complete and runs forever without any progress. Whenever i do a thread dump in this situation, the output is always as follows:
A comparisson of the source and target file in this situation shows: Somewhere in the middle of the target file, a block sized by a multiple of 4k is missing. I observed the following blocksizes: 4k, 8k, 32k. After that block, the content again matches. The target file is always a bit shorter, but not in the same way as the missing block. What i often observe: if a 32k block is missing in the middle of the file, the file is around 55k smaller. The final content of the target file does not match the final content of the source. Maybe this is due to some buffered data not yet written by the sshd?
My test file is about 260MB:
$ seq -w 30000000> testfile
Using testfiles of size ~3MB or ~30MB i was not able to reproduce this problem. I need to transfer the file repeatedly for several hours to run into the problem.Currently i was able to reproduce this problem with the following target configurations: cygwin, OpenSSH_7.2p2, OpenSSL 1.0.2g 1 Mar 2016 cygwin, OpenSSH_7.6p1, OpenSSL 1.0.2n 7 Dec 2017 cygwin, OpenSSH_6.6.1p1, OpenSSL 1.0.1i 6 Aug 2014
On the other hand, i was not able to reproduce it on linux, OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8, OpenSSL 1.0.1f 6 Jan 2014 linux, OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
While it currently might looking like a cygwin problem, couldn't it be a timing/concurrency issue which only occurs due to the limited performance of cygwin?
Please let me know if (and how) i can provide more information.
This is the code i use to perform the scp put: