Using >170Gb of memory and then crashing to download large folders from a server

iterate-ch / cyberduck

Cyberduck is a libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift file transfer client for Mac and Windows.

https://cyberduck.io/

GNU General Public License v3.0

3.27k stars 290 forks source link

Using >170Gb of memory and then crashing to download large folders from a server #15510

Open gabersyd opened 9 months ago

gabersyd commented 9 months ago

This is a bug. I use a MacStudio, with M1 Ultra processor.

Instead of writing the data on my harddrive, it seems that it is trying to store it in the memory of the computer. cyberduck bug

ylangisc commented 9 months ago

Possible duplicate of #15488. What's the protocol you are using? Any chance to follow the steps in https://github.com/iterate-ch/cyberduck/issues/15488#issuecomment-1876761991 and provide a screenshot when reproducing the issue (memory >2GB)?

tristanmcguire commented 5 months ago

This seems to be the same/similar issue to what I have found. Mac M2 Pro, Sonoma 14.4, downloading via SFTP. CyberDuck Version 8.8.2 (41344). At the beginning of the transfer:

After ~15mins:

And after ~30mins:

I discovered the issue only after getting the 'out of application memory' error message and things going a bit wonky. Didn't continue to complete crash, but stopping the transfer, closing and re-opening CyberDuck resets everything and the next download will repeat the process.

ylangisc commented 5 months ago

Thanks @tristanmcguire for the screenshots. As I'm not able to reproduce the issue I rely on the reporters to provide the information needed to find the cause. So, it would be great to also get a VisualVM screenshot from such a situation.

kdambiec commented 4 months ago

I've also encountered this issue quite often on Mac so had to switch from Cyberduck to a different application.

ylangisc commented 4 months ago

I've also encountered this issue quite often on Mac so had to switch from Cyberduck to a different application.

To get a VisualVM screenshot would be very helpful to track down the issue as I'm not able to reproduce the issue.

whwread commented 1 month ago

Same issue here with DropBox. Mac Mini M2, Sonoma. Trying to queue large amounts of data to be repatriated. The Mac Mini will choke itself in the process using all the RAM and about 2x that on the drive as virtual memory until the system runs out of memory. Now using GoodSync, which is 1/3 of the speed. Happy to provide more info if needed.

DerekTCS commented 1 month ago

Same issue here, using 38GB on my M1 Pro. Is there a workaround ? I already tried disabling segmented transfers.

kdambiec commented 1 month ago

Same issue here, using 38GB on my M1 Pro. Is there a workaround ? I already tried disabling segmented transfers.

I haven't found any work around, and the developers appear to be unable to reproduce the issue so I doubt there will be any fix anytime soon.