Open huitema opened 2 years ago
@suhasHere that means, in particular, unifying the Quicrp and Quicr layers. Right now, it is duplicate functionality. Maybe migrate the Quicr code inside the Quicrp prototype. That would be good for performance, and also for control.
@huitema may be I am missing something here, what do you mean by "migrate the Quicr code inside the Quicrp prototype."
My understanding of this work was
What are your thoughts here ?
When we were discussing performance, you told me that your code would read the media segment, format it as a quicr message in a separate thread, and then write it out as a datagram frame. I think it would be much cleaner if that wrapping up in a quicr message was done inside quicrq.
The current send pipeline is encoded media -> fragment -> encode in to app message structutre that has app specific identities -> enqueue the binary blob into the transport queue
=== then on picoquic requesting for new data --> deque the opaque binary blob from the transport queue --> set it as data parameter into the quicrq callback
We can discuss further on how and where to optimize the flow here over a call or something
I think we should discuss this further, as we progress towards a more refined prototype. The more I think of it, the more I believe we need to split the application into an end-to-end layer visible by clients and origins, and a media-transfer layer implemented by relays, clients and origins. We cannot expect the relays to have detailed knowledge of the application, which is why I think we need something like a "media stream" abstraction. This would get a split of functions:
That probably mean revising the QUICR protocol and split it between what belongs in "media stream" and wht in "application control".
I think this points to the biggest "impedance mismatch" between the QUICR spec and the implementation. The QUICR spec assumes that every message is either "subscribe" or "publish". If I read the spec, each piece of data sent on the media stream is a "publish" message, carrying the whole set of protocol data:
PUBLISH {
ORIGIN (String)
NAME (Number128)
FLAGS (Byte)
FRAGMENT_ID (Number16)
BESTBEFORE (Number64)
TIMESTAMP (Number64)
PAYLOAD (ByteArray)
}
In the implementation, I assumed that the full Subscribe or Publish messages are only sent at the beginning of a "media stream", identified by a media stream ID -- typically the ID of the QUIC Stream used for control in the datagram case, or for both control and transmission in the stream case.
So we have a split:
@huitema Agree with you that we need to rethink it a bit. The basic expectation on relays here is
I am with you on the fact that we need to split the things into
Let me setup sometime to discuss the same . thanks
The differences between code and spec started with the need to support both QUIC streams and QUIC datagrams, and also with the need to separate signaling and transport. I think that we can draw a path forward as follow:
@huitema +1 it aligns with our chat and also our thinking towards having a workable demo that shows how relays and clients interact with this protocol.
also I feel, we need to keep streams only mode as well in the spec and have a spec to show how datagrams can be used. Here is the rough list of specs we might end in the initial versions
We could document QUIC streams as well. But we will have to answer some questions, in particular:
1) Is the QUIC stream on which data is published the same as the QUIC stream on which the "subscribe" is received? Currently it is, but that implies a single stream for all group of objects, thus head of line blocking.
2) Do we want a single QUIC Stream for all objects, or one stream per group of objects? One stream per GOO will result in better performance and control, e.g., possibility to drop old group of objects to "skip ahead" if the queue is full. But it is more complex.
3) What is the content of the stream? Are these full objects, or fragments? If full objects, that requires reassembly if a relay receives datagrams and sends streams.
4) Do we send a full header for each object in the stream? Something like
5) Is the content in order? Can relay drop low priority data?
And vice versa, probably.
That may mean developing a way to acquire the manifest, new formats for subscribe messages, etc.