Extend SUBSCRIBE_OK message to include information about the live edge of the track

[wilaw]

Consider an edge relay, receiving and forwarding relative SUBSCRIBE requests for -5, -3, and -1 for the same track. The first object it receives back is group 37. It cannot tell where to place this object in the sequence and must therefore hold it until it knows more information. If the next object is Group 39, it still cannot forward it, because the head sate is undefined.

We can solve this problem simply by having the SUBSCRIBE_OK response indicate the latest group and object number:

SUBSCRIBE_OK { Track Namespace (b), Track Name (b), Track Alias (i), Expires (i), LastestGroup(i), LastestObject(i) }

One consequence of adding these fields is that an edge, which has no active subscription for the track, cannot send the SUBSCRIBE_OK response until it itself receives a SUBSCRIBE_OK after forwarding the SUBSCRIBE. This makes subscription an async process in which the acknowledgment of the subscription may take in some cases a RTT back to the origin. I think this is necessary for stable state management.

[kixelated]

+1 this is required to build a relay and a non-realtime player.

One consequence of adding these fields is that an edge, which has no active subscription for the track, cannot send the SUBSCRIBE_OK response until it itself receives a SUBSCRIBE_OK after forwarding the SUBSCRIBE. This makes subscription an async process in which the acknowledgment of the subscription may take in some cases a RTT back to the origin.

This was required anyway, otherwise the relay would swallow any application SUBSCRIBE_ERROR codes. My mental model is that SUBSCRIBE_OK == HTTP response headers.

[suhasHere]

One consequence of adding these fields is that an edge, which has no active subscription for the track, cannot send the SUBSCRIBE_OK response until it itself receives a SUBSCRIBE_OK after forwarding the SUBSCRIBE

Moqt is a pub/sub protocol and is hop-by-hop protocol. A subscribe ok means the susbcribe request was validated at the relay. A OK isn't tied to getting a OK from the original publisher. Its sub/sub-ok is a async operation ( like any typical pub/sub systems out there). An ok for subscribe means a promise that if there is a matching data ( either in cache or coming in from a live publisher at some point), it will be forwarded .

[suhasHere]

Consider an edge relay, receiving and forwarding relative SUBSCRIBE requests for -5, -3, and -1 for the same track. The first object it receives back is group 37. It cannot tell where to place this object in the sequence and must therefore hold it until it knows more information

Clarifying question , if the use-case is something like VOD or non real-time flows, I think the same can be done by

1. Subscribe to live edge using relative offset and not the latest group/object
2. Subscribe again to the right point by absolute offert

[kixelated]

One consequence of adding these fields is that an edge, which has no active subscription for the track, cannot send the SUBSCRIBE_OK response until it itself receives a SUBSCRIBE_OK after forwarding the SUBSCRIBE

Moqt is a pub/sub protocol and is hop-by-hop protocol. A subscribe ok means the susbcribe request was validated at the relay. A OK isn't tied to getting a OK from the original publisher. Its sub/sub-ok is a async operation ( like any typical pub/sub systems out there). An ok for subscribe means a promise that if there is a matching data ( either in cache or coming in from a live publisher at some point), it will be forwarded .

I absolutely don't want this behavior.

It means that a generic relay will always return SUBSCRIBE_OK, even if it cannot route (or authenticate) a SUBSCRIBE. A typo or finished broadcast would be indistinguishable from a valid request unless OBJECTs start to flow. SUBSCRIBE_OK would be equivalent to HTTP 100 CONTINUE, which is truly useless, and there would be no equivalent to 200 OK or 404 Not Found.

Pub/sub doesn't mean black hole simulator either. I've only used RabbitMQ a long time ago, but if you tried to bind a queue (ie. SUBSCRIBE) to an exchange (ie. ANNOUNCE) it will return an error if the exchange doesn't exist. Is the justification that pub/sub protocols are supposed to black hole, or is this behavior that you actually want?

Instead of hanging indefinitely if a relay can't route a SUBSCRIBE, wouldn't it just be better to receive a 404 Not Found? You're allowed to try again, but now there's no timers or guess work as to if the SUBSCRIBE could be routed to an origin.

[kixelated]

Consider an edge relay, receiving and forwarding relative SUBSCRIBE requests for -5, -3, and -1 for the same track. The first object it receives back is group 37. It cannot tell where to place this object in the sequence and must therefore hold it until it knows more information

Clarifying question , if the use-case is something like VOD or non real-time flows, I think the same can be done by

1. Subscribe to live edge using relative offset and not the latest group/object
2. Subscribe again to the right point by absolute offert

It's for higher latency targets. HLS/DASH clients don't start at the latest segment, but rather start further back based on the target latency.

If I understand correctly, you're saying the relay:

0. has an empty cache.
1. receives a SUBSCRIBE start=relative/3 and SUBSCRIBE start=relative/5.
2. issues issues a live subscription upstream: SUBSCRIBE start=relative/0
3. waits for the first OBJECT to arrive for that subscribe, using it as the live playhead: N = object.group
4. issues a backfill subscribe upstream: SUBSCRIBE start=absolute/N-5 end=absolute/N
5. forwards any objects based on the mapping from relative to absolute.

It works for relays, but at the cost of an extra round-trip to origin. However, it won't work for clients that want to start playback N groups back (jitter buffer size), because the player doesn't know when/what to render without first receiving N distinct groups.

Here's what we would like:

0. has an empty cache.
1. receives a SUBSCRIBE start=relative/3 and SUBSCRIBE start=relative/5.
2. (re)issues a subscription upstream: SUBSCRIBE start=relative/5
3. waits for SUBSCRIBE_OK latest=N
4. forwards any objects based on the mapping from relative to absolute.

[afrind]

Individual Comment:

I think SUBSCRIBE_OK needs to resolve all relative Subscribe Hints that were passed in the Subscribe. Otherwise, I'm really not sure how my library will know when an Object arrives if it belongs to a particular subscription or not -- the OBJECTs that arrive have only track name and absolute group/object numbers. I see the use case for some relays wanting to optimistically SUBSCRIBE_OK, but then there would need to be a follow up message that resolves the relative hints when the relay finds out where they map.

[suhasHere]

I think SUBSCRIBE_OK needs to resolve all relative Subscribe Hints that were passed in the Subscribe. Otherwise, I'm really not sure how my library will know when an Object arrives if it belongs to a particular subscription or not -- t

Couple of observations

• what use-cases would involve same client sending multiple subscription hints at the same time ?

• Hints are read access points on the same track and if a client ask for multiple such points, it will get objects on the same track regardless .. how relays puts them together in such a case for a given client cannot be normatively specified

  @afrind can you give me a concrete use-case example of "resolve all relative Subscribe Hints that were passed in the Subscribe" from a single client so that it helps understand the issue better. I personally don't think the issue being discussed is directly related to this ...

[wilaw]

what use-cases would involve same client sending multiple subscription hints at the same time ?

The use case of an edge relay forwarding simultaneous requests from multiple connected clients. This is a common use-case for any edge or fan-out relay.

[afrind]

Individual Comment:

what use-cases would involve same client sending multiple subscription hints at the same time ?

The use case of an edge relay forwarding simultaneous requests from multiple connected clients. This is a common use-case for any edge or fan-out relay.

This is definitely one use case. Another is a client making two simultaneous subscriptions: eg live edge and range request starting some time in the past.

[fluffy]

One thing that will not scale is every subscription needing to go back to the original publisher. Consider the cases of a million users are joining a session that starts at the top of the hour and that is published from a mobile device. I think that in any real system there is also going to be no single idea of the live edge. The live edge is relative to which relay you subscribed to. If the relay you are on is over a geo sync satellite, it is going to be behind ones that are not.

So here is what I think we should do. We should not allow relative subscription to things in the past. I think the uses cases I have heard for this can be deal with as subscribe to live edge forward and as once the client find out what object that is, do an absolute subscribe for any data in the past to fill in buffers etc. So to be clear, no relative group or object sequences that are negative but still allows absolute. This seems to meet the use cases and remove a ton of edge case design problems.

To be clear, I think we should be able to subscribe to the current group, but give me object 0 forward in that group. If it is a group per GOP, this allows one to get everything needed to decode the current frame.

[wilaw]

One thing that will not scale is every subscription needing to go back to the original publisher.

But not every subscription has to go back to publisher. It only has to go back to a node that is already subscribed and understands the live edge. Furthermore, concurrent requests would be coalesced at each node. Imagine in the worst case that 1 million requests are made within 100ms to a (really powerful) edge. That edge would only forward one request to the origin. All the rest would be held pending that response. This is exactly what happens when a few million people ask for a live HAS stream. Due to fan out and request coalescing, the CDN only sends a few requests back to the actual origin.

[kixelated]

One thing that will not scale is every subscription needing to go back to the original publisher.

But not every subscription has to go back to publisher. It only has to go back to a node that is already subscribed and understands the live edge. Furthermore, concurrent requests would be coalesced at each node. Imagine in the worst case that 1 million requests are made within 100ms to a (really powerful) edge. That edge would only forward one request to the origin. All the rest would be held pending that response. This is exactly what happens when a few million people ask for a live HAS stream. Due to fan out and request coalescing, the CDN only sends a few requests back to the actual origin.

Yeah exactly.

Only the first SUBSCRIBE will propagate upstream and the SUBSCRIBE_OK is used to bootstrap the live edge. The relay can infer the live edge while the single upstream subscribe remains active.

upstream	direction	downstream
SUBSCRIBE start=relative/3	<-	SUBSCRIBE start=relative/3
SUBSCRIBE_OK start=absolute/74	->	SUBSCRIBE_OK start=absolute/74
	<-	SUBSCRIBE start=relative/1
	->	SUBSCRIBE_OK start=absolute/76
OBJECT group=75	->	OBJECT group=75

Because of the SUBSCRIBE_OK, the relay knows that the received OBJECT only gets forwarded to the first downstream subscribe. Otherwise it would have to guess the live edge.

The relay keeps track of max_group while the upstream subscription remains active. The initial value is set by SUBSCRIBE_OK but all future updates are based on received OBJECT messages. This way is can map from relative->absolute and reply with SUBSCRIBE_OK without issuing a new upstream subscription.

upstream	direction	downstream
OBJECT group=78	->	OBJECT group=78
OBJECT group=79	->	OBJECT group=79
	<-	SUBSCRIBE start=relative/1
	->	SUBSCRIBE_OK start=absolute/78

[suhasHere]

For the original use-case reported on this issue, if there are multiple subscribes at a relay with varying relative offsets, the relay has no choice but to send each combination of subscribe and a given -ve relative offsite upstream since that identifies each unique data point in the track.

This has couple of issues

• it doesn't scale across relays and also when there are very high rate of subscriptions across relays and clients
• it makes the lookup and state management complicated
• latencies across relays and client locations ends up producing varying results for the same query since it depends on when the query was made and how long would it take to
• for conferencing cases when there are multiple publishers who can be active (have live edge), unjoined yet ( so no live edge) , the expectation for subscribes to get a OK back from original publishers will not scale in terms of latency of joins

The more i think of it, subscribing back into paste using a relative offsets and expecting the answer to be vetted by the original publisher "unnecessarily complicates the protocol".

[wilaw]

if there are multiple subscribes at a relay with varying relative offsets, the relay has no choice but to send each combination of subscribe and a given -ve relative offsite upstream since that identifies each unique data point in the track.

The group sequence numbers are consistent between subscriptions . If a relay receives relative subscribes of -5,-3 and -1, all it has to is send a single subscribe upstream of -5. The response would give it the live edge group as well as all the content for the various subscriptions. If the subscriptions were -345, -2, -1, the relay could decide to make two subscriptions upstream - one for -345, the other for -2, because it didn't want to force the smaller relative subscriptions to wait for the larger one. The point is, this is optimization logic for the relay. It does have a choice in what subscriptions it sends upstream based on the timing and diversity of the input subscriptions.

the expectation for subscribes to get a OK back from original publishers will not scale in terms of latency of joins

Why is the latency of the join affected? Consider a subscriber talking to an edge where the subscriber is 100ms away from the origin. We consider two scenarios: in the first case the subscriber receives a SUBSCRIBE_OK immediately after subscribing but then waits 100ms for the audio and video to arrive from the origin. In the second case the the subscriber waits 100ms to receive SUBSCRIBE_OK but it is immediately followed by the audio and video. So in both cases the subscriber gets its audio and video after 100ms. The media latency is consistent under either scenario.

[suhasHere]

The group sequence numbers are consistent between subscriptions . If a relay receives relative subscribes of -5,-3 and -1, all it has to is send a single subscribe upstream of -5.

If the relay logc is to send only a single subscribe upstream, then you don't need anything to be said in Subscribe OK either. Since when the first object arrives it can compute the live edge by adding 5 to the object being reported and the compute the absolute values for others (in this case for -3 and -1). That would definitely make things simpler for sure.

[suhasHere]

the relay could decide to make two subscriptions upstream - one for -345, the other for -2

For cases where a client wants to go back 345 groups in past, i will be very much inclined to do an absolute request after learning the latest known state of the group as an end application OR something catalog can describe more information to make the right absolute request on the first go

[wilaw]

Since when the first object arrives it can compute the live edge by adding 5 to the object being reported and the compute the absolute values for others (in this case for -3 and -1).

It can't do this because the order of items it receives as Streams is not guaranteed. The origin may send the groups in order -5,-4,-3,-3,-1, but they could be received at the relay (or any hop) as -3, -5,-4,-1,-2. That's why you need an explicit notification from the origin as to where the live edge is. Only that plus the group sequence numbers allow the correct sequencing, not the order of arrival.

[suhasHere]

In this case there was just a single subscribe request with -5 was sent upstream, based on your explanation above. This avoids all the confusion which would happen when there are multiple inflight -ve offsets.