search

storj-archived / core

Deprecated. Implementation of the Storj v2 protocol for Node.js.
https://storj.io
Other
395 stars 88 forks source link

Storj will take a long time to download/upload the last shards #705

Closed ghost closed 6 years ago

ghost commented 7 years ago

This is happening when using libstorj v 1.0.1.

Expected Behavior

Downloading and uploading should happen as fast as possible.

Actual Behavior

Up- and downloads will become slow towards the end.

Steps to Reproduce

Up- or download a (large) file.

Additional comment:

Bittorrent uses a mechanism where, when only a few chunks are left to complete the torrent, it will download those simultaneously from multiple sources, and as soon as one of them completes, the others are cancelled. Maybe such functionality could be added to storj, preventing downloads to be throttled by a single slow farmer.

braydonf commented 7 years ago

Downloading the same shard is something we've discussed. We would need to be able to download from specific positions of a file. This improvement would be best described in a SIP (https://github.com/Storj/sips) since it will affect multiple repositories from client side such as libstorj (https://github.com/Storj/libstorj/ ), bridge (https://github.com/Storj/bridge), to farmer side it storjshare (https://github.com/Storj/storjshare-daemon).

braydonf commented 6 years ago

We should track this on https://github.com/Storj/libstorj

braydonf commented 6 years ago

Moved to https://github.com/Storj/libstorj/issues/411