Segmented downloads with multiple connections per file

[cyberduck]

20868bf created the issue

Cyberduck does not seem capable of multi-segmenting files in FTP transfer. Consequently, for servers using this method of transfer, files being transferred via Cyberduck do not make full use of bandwidth and transfer far slower than they should. Although I have donated to Cyberduck development because I believe it is a very nice product, I found that for my needs, a product called Speed Download by Yazsoft does provide for multi-segmented transfers at far faster speed.

[cyberduck]

@dkocher commented

Actually a duplicate for #1135.

[cyberduck]

6caada9 commented

Segmented download is not the same as multi-connections (#1135) though it does depend on it... With segmented downloads a single large file is downloaded in several parts each using a separate connection. Multiple other download clients now support this - will CyberDuck support this in future?

[cyberduck]

@dkocher commented

Duplicate for #1135. We have implemented multiple connections per transfer but it is not currently enabled by default.

[cyberduck]

6caada9 commented

lol - this is explicitly NOT the same as #1135! If you don't want to do it that's your prerogative but at least take the time to understand the request!

[cyberduck]

6caada9 commented

https://whatbox.ca/wiki/Multi-threaded_and_Segmented_FTP is a good description of the difference.

[cyberduck]

@dkocher commented

Replying to [comment:6 ajostergaard]:

lol - this is explicitly NOT the same as #1135! If you don't want to do it that's your prerogative but at least take the time to understand the request!

Sorry I was just skimming over and not reading properly.

[cyberduck]

1ab0d44 commented

A thumbs up for implementing segmented downloads. -Parallel threads downloading different parts of the same files which are assembled when the download has completed - enabling a 100mbit connection (becoming more frequent) to max out the transfer speed rather than be limited by latency.

[cyberduck]

@dkocher commented

In efb9fc6ca2845871abd8e4d6e99710ea93cb35de. The feature is not currently enabled by default. To enable, set the hidden settings queue.download.segments.threshold=0 and queue.session.pool.size=2 for two concurrent connections. When using the CLI run with env "queue.download.segments.threshold=0" env "queue.session.pool.size=1" duck ….

[cyberduck]

6caada9 commented

Thank you! :)

[cyberduck]

1ab0d44 commented

Fantastic that the feature is now in the code, thank you so so much. Would it be possible to add access to it via the GUI ?

[cyberduck]

@dkocher commented

Replying to [comment:14 bobthebuilder]:

Fantastic that the feature is now in the code, thank you so so much. Would it be possible to add access to it via the GUI ?

We will have to let the new feature bake for 4.7 until we will make it available through the user interface.

[cyberduck]

@dkocher commented

Make sure to use e6dcee6bacb670711b852dd804fac9e940da57e3 which includes an important fix.

[cyberduck]

@dkocher commented

As of 9deb42f1ba0efaf0d7a9ac9cc52976b0149ad8e9 use the setting queue.transfer.type.enabled to enable the option for concurrent connections in the Preferences → Transfers → Transfer Files… and the bookmark settings.

defaults write ~/Library/Preferences/ch.sudo.cyberduck.plist queue.transfer.type.enabled "newconnection browser concurrent" 

This will add the option Open multiple connections. Use the setting Number of concurrent transfers to control the number of parallel connections.

[cyberduck]

@dkocher commented

Enabled in 5aeeb691a5ce2daa61fec64e49616373e877078a.

[cyberduck]

@dkocher commented

Disabled by default in 02ef4f82dc6fc7a7f581e5532d39c821e8a4a139. Set the hidden configuration option queue.download.segments to true to enable.

[cyberduck]

@dkocher commented

Reopen this issue to enable by default and add configuration option.

[cyberduck]

@dkocher commented

Milestone renamed

[cyberduck]

@dkocher commented

In 4f748a8a005d816cb56adad2f7a2f537e67e8762.