Open reit-c opened 9 years ago
qqueue
commented
8 years ago The Peer-to-Peer Streaming Protocol (PPSPP) made it to RFC status:
https://tools.ietf.org/html/rfc7574
It's by the https://github.com/libswift/libswift and https://github.com/Tribler/tribler guys. I feel like there are enough political differences in ipfs to prevent a fully compatible implementation, but the ideas are sound.
jbenet
commented
8 years ago We're not opposed to using PPSPP at all-- in fact, i think it's great to see it reach RFC status :clap: :clap:
In our case, it would be another libp2p module-- and most of the time we wouldnt be able to eek out as much perf out of it as other protocols, but all for it.
danimesq
commented
5 years ago @reit-c @qqueue @jbenet Updates?
Stebalien
commented
5 years ago
Per the IRC discussion yesterday, I believe the concept of real-time streaming (e.g. of video) to be quite doable and in line with the basic principles of IPFS, and may not require much in the way of alterations to anything to get it working.
Working with video as the example, we might traditionally have a video chunked into a number of blocks, tied together by a root node linking them all together.
This is a normal and intuitive dag structure, but not the only possible one. Another format might take the form of a linked list of sorts, with each item in the list represented by a node linking to two others, the first containing the associated data and the other being the next item in the list.
Here, the video blocks can still be accessed by traversing the tree from the root, albeit in reverse order. However, this structure is extensible; new video chunks can be layered on top to add to the end of the video without drama. If combined with IPNS, this would allow for arbitrary data streams by adding a new item to the end of the list and resetting the DHT to point to the new end each time.
However, this alone does not provide for real-time streaming capabilities, as currently IPNS is DHT based and slow to update. This may in itself have its uses (logging systems, incremental backups etc.) but to achieve real-time updates a different system is necessary.
A pub/sub system is planned (https://github.com/ipfs/ipfs/issues/73#issuecomment-107242443) which may, depending on its design, solve this issue 'for free'. Under this system, IPNS naming updates would be pushed directly to whatever devices subscribed to that name. In the case of video streaming, the streamer could (perhaps once a second) create a video chunk, hash it, add it to the tail of the linked list, and publish the new root out to IPNS. Subscribers would then effectively receive a video stream, and any new subscribers joining partway through would be able to rewind to the beginning of the stream at will (that is, by traversing the tree). The stream would be self-archiving, and when ended could still be accessed at any time in full via the final root hash/associated IPNS entry (so long as even one user still cares about it). The system would also be fault-tolerant; in the event a publish was somehow dropped, the next publish would recover it by way of including it in its dag structure, no data would ever be lost.
How effective such a system could be depends on the pub/sub implementation. A good approach might involve avoiding the DHT and having the subscribers keep connections between each other alive, such that as new stream data comes in it can be immediately redistributed between the subscribers without having to renegotiate connections each time. Standard bitswap (with sane reciprocation algorithms) might suffice for the actual video content block transfers, but it would be important to design the subscription model such that the publisher doesn't get lumbered with having to send dag structure updates to potentially many thousands of subscribers each second. For the dag structure updates themselves something resembling a peercasting arrangement may be effective (https://en.wikipedia.org/wiki/Peercasting).