Huge speed drop when file size exceeds around 300-500 GB over SSHFS

[knaccc]

I have been trying to copy a 2 TB file to a local gocryptfs mount, which is backed by an SSHFS mount to a Linux NAS on an internal network.

It starts out at 50MB/s+. However, I’ll check back on the transfer after a few hours, and observe that the speed has dropped to 2-3MB/s or below.

I tried half a dozen times over the course of 2 weeks, and the same thing happened each time. It'll be fast at first, but then I'll return and see it stuck at around 300-700GB for days because it has become cripplingly slow for the remainder of the copy.

I then tried copying the file to the SSHFS mount directly, without using gocryptfs. This completed perfectly, at an average speed of 70MB/s+.

There is no problem when I create a fully local gocryptfs mount, where the storage is on a local SSD. The problem only occurs when the cipher folder is an SSHFS mount.

There must be something about the way gocryptfs works that would cause this slowdown for huge file sizes - any ideas please?

Thanks

[mbgevers]

I have been trying to copy a 2 TB file to a local gocryptfs mount, which is backed by an SSHFS mount to a Linux NAS on an internal network.

It starts out at 50MB/s+. However, I’ll check back on the transfer after a few hours, and observe that the speed has dropped to 2-3MB/s or below.

I tried half a dozen times over the course of 2 weeks, and the same thing happened each time. It'll be fast at first, but then I'll return and see it stuck at around 300-700GB for days because it has become cripplingly slow for the remainder of the copy.

I then tried copying the file to the SSHFS mount directly, without using gocryptfs. This completed perfectly, at an average speed of 70MB/s+.

There is no problem when I create a fully local gocryptfs mount, where the storage is on a local SSD. The problem only occurs when the cipher folder is an SSHFS mount.

There must be something about the way gocryptfs works that would cause this slowdown for huge file sizes - any ideas please?

Thanks

I assume the Problem is not gocryptfs but more SSHFS

SSHFS is known to have certain limitations and to become slow under certain circumstances.

Are you sure you never hit the issue without gocryptfs?