atomashpolskiy / bt

BitTorrent library and client with DHT, magnet links, encryption and more
https://atomashpolskiy.github.io/bt/
Apache License 2.0
2.42k stars 381 forks source link

In-memory storage with streaming output #9

Open atomashpolskiy opened 7 years ago

atomashpolskiy commented 7 years ago

In-memory storage by itself might be useful in a number of scenarios:

All of this might be useful, but the real killer feature would be an optional "streaming" mode of operation. In this mode the storage would simply re-route the incoming data to a binary output stream. Of course, this mode will only make sense, when the sequential piece selector is used, and will require accumulating and re-ordering the arriving blocks under the hood.

One possible usage scenario would be to add an option for CLI client to stream the torrent data to stdout. Then this data can be piped to any other process, e.g. for large textual files it might be less, tail or grep. For video/audio it will be a media player that supports live stream decoding.

magJ commented 7 years ago

If you implement a storage backend using Java 7's java.nio.file classes, you can get in-memory storage working for free by using jimfs

atomashpolskiy commented 7 years ago

Also see this thread on HN, considering caching of torrent's data. Probably need to move this to a separate issue.

atomashpolskiy commented 7 years ago

See discussion in #21 for some background