Pure memory storage provider

mafintosh / torrent-stream

The low level streaming torrent engine that peerflix uses

MIT License

1.94k stars 227 forks source link

Pure memory storage provider #152

Open ghost opened 8 years ago

ghost commented 8 years ago

Hello,

I watch your project and i want to say I really appreciate what you do guys. I want to implement in memory storage.

That means i want to store only pieces that i will use for playback, maybe i can allocate 50mb for future pieces and 50mb for pieces that was already played by vlc (for seeding).

I familiar with torrent stream code, but still want to ask, what you can recommend me to start?

@mafintosh @feross

Ivshti commented 8 years ago

I propose those ideas:

creating a pure memory storage in place of fs-chunk-store
implementing more events/hooks into torrent-stream so we can:
implement a buffer control, maybe as a separate module, that ensures we keep no more than 100mb in memory at once; the last recently used pieces will be freed, but this also has to emit events to torrent-stream so that it knows to adjust have

Also, there may be something already started on this, we should check the other issues

UPDATE there's this https://github.com/mafintosh/torrent-stream/issues/101 But it doesn't satisfy the requirement of being limited to N amount of pieces. This won't happen easily without adjusting other memory structures, e.g. bitfield of pieces we have

ghost commented 8 years ago

@Ivshti yes, i saw this implementation, but there is no any limits of pieces.

feross commented 8 years ago

There's already memory-chunk-store, but it doesn't throw away old pieces like you want.

The problem with doing that is that as pieces are downloaded, the client sends HAVE messages to all connected peers to inform them that you have new pieces. There's no corresponding message to inform them that you deleted the piece, so what should be done when they request a piece that you've deleted? It's not clear.

ghost commented 8 years ago

@feross Yes, this is not clear, and this is problem. But what will happen if i will don't send HAVE message? Will peers bans me?

feross commented 8 years ago

Not sure, but it's likely that they'll disconnect from you, or stop uploading to you. At the very least, you'll be sending them less data, which decreases your upload speed, makes you appear a worse peer to them, and thus gets you a slower download speed from them.

ghost commented 8 years ago

@feross thanks for help. i understand, but i don't have any other ways, because i need to try

Rogue85 commented 5 years ago

There is some progress in this case?