Catalog track definition insufficient to communicate client selection criteria

[wilaw]

The current CATALOG allows for an overly simplistic track definition

Track Descriptor { Track ID (i), Container Format (i), Container Init Payload (b) }

These fields do not include sufficient semantics to communicate selection criteria present in modern streaming productions. Some examples would include:

• language - audio tracks in english, french german
• accessibility options - video description, signing
• captions
• subtitles
• codec - HEVC, AVC, VVC
• resolution - 1080p, 2160p,
• framerate - 30,60,24
• spatial audio - 5.1, Dolby Atmos

[vasilvv]

That is by design. All of those are available in the Container Init Payload, which in case of fMP4 would contain a MOOV atom.

[suhasHere]

This assumes fMP4 as the only container format and supports a specific set of use-cases with specific set of constraints

[vasilvv]

Indeed, if you are defining a format that is not fMP4, you would need to figure out how to specify all of those details for the newly defined format.

[suhasHere]

That's true ... but I am not seeing a reason why CATALOG message define any structure .. All it needs is

CATALOG Message {
  WarpStreamid/TrackId
 [Container Type(8)],
  Length(i),
  Payload(...)...
}

please note the catalog payload might be encrypted too and its contents is not useful for intermediaries .

[kixelated]

That's true ... but I am not seeing a reason why CATALOG message define any structure .. All it needs is

CATALOG Message {
  WarpStreamid/TrackId
 [Container Type(8)],
  Length(i),
  Payload(...)...
}

please note the catalog payload might be encrypted too and its contents is not useful for intermediaries .

Yep that's the idea, and is almost verbatim in the draft. The only difference is that a CATALOG message can have multiple tracks, but each Track Descriptor is exactly as you described.

I like the idea of splitting each track descriptor into a separate message (#43 had the TRACK message) so tracks can be added/updated/removed independently. The CATALOG message with all tracks was a compromise.

[kixelated]

The current CATALOG allows for an overly simplistic track definition

Track Descriptor { Track ID (i), Container Format (i), Container Init Payload (b) }

These fields do not include sufficient semantics to communicate selection criteria present in modern streaming productions. Some examples would include:

• language - audio tracks in english, french german
• accessibility options - video description, signing
• captions
• subtitles
• codec - HEVC, AVC, VVC
• resolution - 1080p, 2160p,
• framerate - 30,60,24
• spatial audio - 5.1, Dolby Atmos

Like Victor mentioned, he realized that most of this stuff was already in the fMP4 init segment (trak box) and it was redundant to include in the CATALOG. The relay shouldn't parse any of this information so it's nice to delegate it to an opaque "container format" blob that only the broadcaster/viewer will support.

I think all of your examples are present in trak but there's a few things missing in fMP4 that might be useful, like peak bitrate, and we should discuss how to add those fields. Modify fMP4? Add a new track format? Add additional fields to the CATALOG message?

I'd love to see us specify more track formats. A possible example would be "HLS", where the Container Init Payload would be a EXT-X-STREAM-INF string and the Object Payload would be a TS segment (including SPS/PPS). And possibly rename "container" to something else ("track format"?) so it's less confusing.

[wilaw]

Label for Ted: Architectural

[wilaw]

The resolution of this issue depends on two other issues - specifically Relax the CATALOG definition to decouple the base protocol from the streaming format moq-wg/moq-transport#66 and Move Catalog details to its own spec moq-wg/moq-transport#85. If either of these issues is adopted, then the track definition will be a function of the CATALOG description that may or may not reside in a separate spec.

If neither issue is adopted, then we need to address how to extend the the current WARP draft track definition.

here's a few things missing in fMP4 that might be useful, like peak bitrate, and we should discuss how to add those fields. Modify fMP4? Add a new track format? Add additional fields to the CATALOG message?

@kixelated - modifying fmp4 via ISO/MPEG will take a long time and is not a process we would want to follow for ad-hoc changes to selection criteria. The more flexible path for MoQ would be to extend the track format and/or CATALOG message fields to hold the new selection criteria. While we could hang new fields off of CATALOG, they would increase with application requirements. Consider for example a live VR game in which the client subscribes to streaming mp4 texture maps based on spatial location, velocity and degree-of-invisibility (i'm making that up to illustrate my point). We could add TRACK.location, TRACK.velocity and TRACK.invisibility. This could take a year or so to push through IETF. Then we'd need to wait for relay providers to pick up those changes and deploy support for them. That could take another year. That's a long delay before the game developer could use MoQ to stream their game in production. And that's just one application. What if there were ten different ones that all needed different fields in the CATALOG. After years of updates, we'd have 50 or more custom fields in TRACK catering to different applications. So a more flexible approach would be define a standardized key/value selection field for TRACK for carrying application-specific selection criteria not contained within the INIT object. For example TRACK.SELECTION = [{key:value}]. This could be used by all applications for custom selection and it would not require any spec updates.

[afrind]

Catalog track definitions are being moved out to https://github.com/wilaw/MoQ, so maybe we can close this?

[afrind]

@wilaw do you have this issue captured in your repo and can you close this out?