search

mafintosh / torrent-stream

The low level streaming torrent engine that peerflix uses
MIT License
1.94k stars 228 forks source link

Question: torrent-stream performance compared to libtorrent #100

Open 0x4139 opened 9 years ago

0x4139 commented 9 years ago

how is torrent-stream performance compared to libtorrent, i mean writing bindings to libtorrent would make it faster than torrent-stream?

asapach commented 9 years ago

IMHO, any comparison would be unfair. Torrent-stream solves a very specific problem that libtorrent doesn't address: streaming files while they are being downloaded.

Having said that, I find that t-s is not very memory-efficient, but incredibly fast and resilient.

Ivshti commented 9 years ago

As a person who has worked with both to create a product, I can say this: torrent-stream beats libtorrent in terms of speed of download almost every time. By almost I mean that torrent downloading has a late number of variables and a lot of conditions play a role.

That being said, libtorrent usually over time develops a higher speed of download - it has a better peer selection system over time.

Also, libtorrent would have a lower CPU and memory usage. There are several reasons for this. First, we have the barebones c++ vs JavaScript/V8 reason. Second, and more importantly - the architecture/design - libtorrent is designed for downloading in the background vs torrent stream being designed to accelerate fast and keep downloading in order which is more demanding, requires more data kept in memory and more calculations.

Ultimately, different libraries, both very well designed and implemented but for a specific purpose.

0x4139 commented 9 years ago

@asapach can't we achieve the same thing by writing a storage that caches the results in the memory like t-s does?

asapach commented 9 years ago

@0x4139, your premise is incorrect: t-s doesn't cache the whole torrent in memory - just the pieces that are being currently downloaded (like most bittorrent implementations do). What distinguishes t-s is that it is able to prioritize the the pieces that are being requested by the consumers and stream them sequentially (instead of downloading them in random order) while blocking the stream until the pieces are ready.

0x4139 commented 9 years ago

@asapach i didn't say that it caches the whole torrent in memory, you can do that with libtorrent as well. http://www.rasterbar.com/products/libtorrent/manual.html#piece-picker. Don't get me wrong not promoting libtorrent, just want to understand better what is the difference exactly .

asapach commented 9 years ago

OK, imagine that you have a torrent with 100 files and you want to download the 42nd file as fast as possible. I'm not sure what kind of API libtorrent exposes, but judging by the clients (e.g. Deluge), you'd first have to deselect all the other files, or set the highest priority to the required file and wait for it to download (regardless of strategy you pick). With t-s you can just start streaming it and t-s will provide you with the first bytes as soon as the first piece becomes available, before the file download completes.

nevercast commented 9 years ago

Sorry to piggy-back on this question but this does feel to be on-topic. Does 'storage' stream the piece from the file instead of memory as soon as it is complete? Assuming a piece was downloaded before the stream needed to read it or it was already downloaded from overlapping streams?

I'm trying to get a gauge of what every part of this project does and it seems like storage handles reading/writing to File IO, does it handle the pieces that are being downloaded also (As a cache)?

asapach commented 9 years ago

Storage is responsible for storing the downloaded pieces (hence the name). It can be backed by the file system, or by memory, or by carrier pigeons if you implement the contract and pass it as the storage option. Pieces that are in-flight (not downloaded yet), are not being stored.