search

pkg / sftp

SFTP support for the go.crypto/ssh package
BSD 2-Clause "Simplified" License
1.49k stars 379 forks source link

Connection lost when using library with limited bandwidth #566

Closed uncleDecart closed 5 months ago

uncleDecart commented 7 months ago

Greetings :wave:

I'm trying to use your client and test it in low-bandwidth network and I get similar behavior: during download it randomly stops with connection lost error. Here is the code I'm using and sftp library is v1.13.6

In order to artificially limit my adapter bandwidth I'm using wondershaper. 

sudo wondershaper -a eth0 -d 64 -u 64

And during download it returns error : connection lost. To my best understanding, only parameter I have to tune closing connection is Timeout parameter in ssh.ClientConfig and I set it to 0. To my best knowledge, there is no limitation anywhere in SFTP or SSH specification on network bandwidth. I'm downloading files from SFTP server I deployed on Azure.

I saw this issue https://github.com/pkg/sftp/issues/424 , but I'm not sure if it's related to my case

Do you have any points or leads why this is the case for me?

Thanks in advance!

puellanivis commented 7 months ago

As long as you’re using the most recent release, you should be fine from what was going on with #424 . I’m not sure where this connection lost error could be originating from, as we don’t generate it ourselves. You are right there should be no limitation on SFTP or SSH for poor network bandwidth. 🤔 The only thing I can think is that some sort of proxy in the middle is cutting off long-lived connections?

uncleDecart commented 7 months ago

@puellanivis I was able to narrow it down to this line generating error after downloading 45-47 MBytes from file. Since I'm deploying on Azure I can't really say what's happening there. I tried with locally deployed SFTP server and it worked like clock. I'll leave this issue open for some time if I get some troubles to update this issue for anyone struggling with the same problem.

puellanivis commented 7 months ago

Well, the only way that line of code is coming is if it were generated in the call to io.ReadFull(r, b[:length]) which is entirely outside of anything we’re doing…

I think it’s something to do with the Azure network implementation. Possibly even a firewall rule to prevent extremely slow requests, which is a way of DDOSing some servers, with lots of requests that are transferred extremely slow, exhausting request resources, without needing lots of network bandwidth.

tejaskumark commented 6 months ago

@uncleDecart If you have access to azure server, then you can take packet capture on your client and azure server, and match TCP reset is being sent from either client or server, if not then it must be firewall in between resetting connection by sending TCP reset to both party. I have seen this many times in real world scenarios that firewall resetting connection due to IT policies set on it.

uncleDecart commented 5 months ago

Thank you very much for your help and suggestions @puellanivis , @tejaskumark ! I'm closing this issue, because problem was on a server side.