mitchmindtree
commented
6 years ago Research
- RTP: Real-time Transport Protocol - facilitates the transfer of real-time data. Information provided by this protocol include timestamps (for synchronization), sequence numbers (for packet loss and reordering detection) and the payload format which indicates the encoded format of the data. Original RFC.
- RTCP: RTP Control Protocol - provides feedback on the quality of service (QoS) in media distribution by periodically sending statistics information such as transmitted octet and packet counts, packet loss, packet delay variation, and round-trip delay time to participants in a streaming multimedia session. An application may use this information to control quality of service parameters, perhaps by limiting flow, or using a different codec.
- RTSP: Real Time Stream Protocol - used for establishing and controlling media sessions between end points. Clients of media servers issue VCR-style commands, such as play, record and pause, to facilitate real-time control of the media streaming from the server to a client (Video On Demand) or from a client to the server (Voice Recording).
This stackoverflow comment provides a nice breakdown of how these three protocols could be used together for generalised realtime streams.
Thoughts
RTCP seems like it may be unnecessary for many creative coding applications. The primary purpose seems to be monitoring quality of service, useful for making adaptive streams which can account for changes in quality or congestion in the network. That said, it could be useful for larger, more sophisticated setups.
RTSP also may be a little too high-level for what we're looking for - it seems to be a more opinionated set of messages of high-level control like "get parameter", "set parameter", "play", "pause", "record", etc - many of which are irrelevant for many kinds of realtime streams, but may be more useful for controlling remote setups (e.g. over the non-local internet).
One major benefit of using the protocols above is that they are widely used standards. As a benefit, we may not only unveil a simple way of establishing realtime streams, but also unlock interaction with many other applications that already implement these protocols.
freesig
It would be incredibly useful to have a general way of establishing realtime streams of data over a network. Use cases might include streaming audio, video, laser data, lighting data, etc over a network without requiring the need for proprietary or 3rd party software.
It seems that there are several common requirements between different kinds of realtime streams:
TCP_NODELAYflag we can disable Nagle's algorithm, making TCP far better suited for realtime streaming.I can picture a low-level realtime stream protocol which handles each of these lower level properties, allowing developers to build higher-level stream APIs (e.g. audio, laser, video) with far greater ease on top.
Perhaps a protocol like this already exists? It would be great if there was already a protocol like this in use that we could provide an implementation for. If not, I imagine it would not be too difficult to implement something like this. Perhaps a crate along the lines of
rttcp, standing for "Realtime TCP".Background
Recently I have been thinking about beginning work on an open-source, free networked audio crate inspired by Dante, but with an MIT license and less of the arbitrary restrictions. I had not considered working on my own networked audio until writing the ether-dream crate (a protocol for interacting with a networked laser DAC). Seeing how the layout of laser data is so similar to audio data, it made it much clearer to me on how it might be possible to work on a real-time audio crate. After thinking a little more on the topic I started wondering whether a more generalised, "realtime" data stream crate might be a better approach.