3.2.15. Alternate group

[acbegen]

Alternate tracks SHOULD have matching framerate [Section 3.2.24](https://www.ietf.org/archive/id/draft-wilaw-moq-catalogformat-01.html#framerate) and media time sequences.

Different video representations can easily have different frame rates. I don't see the reason for this "SHOULD."

[wilaw]

While they certainly can have different framerates, the days of having the lower renditions at 15fps to save bandwidth are mostly passed by. I view of a lot of CMAF-based production manifests and playlists and the majority today use matching framerates . Constraining alt tracks to have a matching framerate makes player audio/video sync simpler .

[suhasHere]

Also to note, matching framerates will enable changing groups across alt groups natural too.

[acbegen]

Constraining alt tracks to have a matching framerate makes player audio/video sync simpler .

Maybe, but frame rate changes do not require decoder re-initialization. So, I view this as an unnecessary constraint.

[scottlamb]

Constraining alt tracks to have a matching framerate makes player audio/video sync simpler .

Could you explain more please? I'm trying to understand and not getting it. I'm assuming the "framerate" mention is only relevant to video tracks, and alternate tracks having the same frame rate then is meant to ease switching from one video track to another. And it looks like they're meant to be independent (not SVC layers or something). The player needs to start the new track at a key frame (I'm ignoring periodic infra refresh...), so I don't see how non-key frames coming at the same rate/time is helpful.

In fact, I assume the player will switch at a group boundary, since that's a key frame and a join point. So matching group boundaries makes sense; and I see something to that effect in 3.2.15:

Tracks with the same [render] group number SHOULD be rendered simultaneously, are usually time-aligned and are designed to accompany one another.

What I don't see is a way to say "end my subscription to track old and start my subscription to track new at the same group number"; it doesn't seem obvious to me you can achieve that with UNSUBSCRIBE SubcribeId=$OldTrackSubscription ("Objects should stop being sent as soon as possible") + SUBSCRIBE ... with no waste. But I suppose that's a matter for the transport spec. [edit: just saw that an SUBSCRIBE_UPDATE has been added since draft-ietf-moq-transport-03. Still not "atomically switch at an upcoming group boundary" but picking the some group number n later than it's yet seen, narrowing the old subscription to < n, and starting the new at n seems better than using UNSUBSCRIBE anyway.]

[wilaw]

The player needs to start the new track at a key frame (I'm ignoring periodic infra refresh...), so I don't see how non-key frames coming at the same rate/time is helpful.

The frame rate for the decoder is typically set in the decoder via the init segment. All segmented media players (and likely moqt players) will re-initialize as they switch tracks, so therefore it should be possible to switch video framerates between tracks. I guess the hesitation in allowing this stems from the last 10 years of fragile ISOBMFF decoders. I think we can afford to be more forward facing with moqt and not inherit this legacy concern, and therefore I would support removing the constraint that Alternate tracks SHOULD have matching framerate.

What I don't see is a way to say "end my subscription to track old and start my subscription to track new at the same group number"

The API for subscriptions, as you point out, is defined by the moqt spec and not by this catalog spec. The purpose of the catalog is to provide the player all the information it needs to select a track for subscription and to initialize and encode the content that it will receive. The mechanics of that subscription are defined by moqt. The logic of when to switch will be defined by a streaming format spec, such as WARP.

[edit: just saw that an SUBSCRIBE_UPDATE has been added since draft-ietf-moq-transport-03. Still not "atomically switch at an upcoming group boundary" but picking the some group number n later than it's yet seen, narrowing the old subscription to < n, and starting the new at n seems better than using UNSUBSCRIBE anyway.]

Yeah, we debated having an explicit SWITCH (trackFrom, trackTo, groupID) command, but it was deferred in favor of a lower level SUBSCRIBE with startGroup and endGroup as well as SUBSCRIBE_UPDATE. But if there are other fans of SWITCH() (I am one of them) then we should speak up and file an issue.