search

veovera / enhanced-rtmp

This industry-sanctioned project introduces significant enhancements to the RTMP and FLV specifications, outlining advanced features aimed at revitalizing and modernizing the RTMP solution.
https://veovera.org
Apache License 2.0
272 stars 24 forks source link

connect command fourCcList is video specific but doesn't have "video" in its name #10

Closed zenomt closed 4 months ago

zenomt commented 1 year ago

in the "Extending NetConnection connect Command" section, there is a new property "fourCcList". the description says it is an array of strings each "representing a supported video codec". the current Enhanced RTMP document is only extending video codecs; however, one would assume (from #2) that audio is coming soon.

with the name "fourCcList", one would presume that in the future, audio and video fourCCs would be intermingled in that array. this might be fine for some applications, but may complicate the semantics if it's important to tell the difference between "is there an audio codec i've never heard of" and "is there a video codec i've never heard of".

since today the audioCodecs and videoCodecs bitmaps are already separate, for symmetry i propose that the arrays of fourCCs for video and audio also be separated, and that they have names indicating which they're for. that way, if the difference is important to an application, it is at least available.

veovera commented 1 year ago

Good point, I am leaning towards making the requested change. One concern, even though the document is officially beta I am wondering if anyone will be affected by the name change...

zenomt commented 1 year ago

since this is sent by clients to a server, and the proposal is to have distinct names for video and audio, a server could be backwards compatible with this beta spec by understanding fourCcList as well as the new video & audio ones.

veovera commented 1 year ago

since this is sent by clients to a server, and the proposal is to have distinct names for video and audio, a server could be backwards compatible with this beta spec by understanding fourCcList as well as the new video & audio ones.

A later client talking to a server that was just release with enhanced rtmp support and a that only understands 'fourCcList' property name will still be an issue. Server will need to be updated, which it probably will once more codecs get introduced.

veovera commented 1 year ago

In thinking about this some more. I am somewhat hesitant to change the name of fourCcList property to avoid any future confusion. But looking at the proposal in #15 we could perhaps solve this by introducing some special characters. Ex.

  1. "*" - To signal that the client can receive any codec.
  2. "a" - To signal that the FourCC values which follow this entry in the the fourCcList identify audio codecs
  3. "v" - To signal that the FourCC values which follow this entry in the the fourCcList identify video codecs. This is the default value which is optional when the list starts with video codecs.
  4. "a*" - To signal that the client can receive any audio codec and to optionally state which audio codecs it can decode
  5. "v*" - To signal that the client can receive any video codec and to optionally state which video codecs it can decode

This style of signaling is future proof and avoids renaming fourCcList property or adding new ones based on the codec type.

zenomt commented 1 year ago

i have two concerns with the above:

  1. traditional audioCodecs and videoCodecs are essentially sets, and today's fourCcList can also be considered a set, where order doesn't matter. if the list is now ordered with delimiters (that is, a and v to indicate the following fourCCs up to the next delimiter are audio or video, or a set if there are no delimiters or up to the first delimiter), that is a significant increase in semantic complexity.
  2. if we're going to stay with a unified fourCcList instead of having separate ones for audio and video, i like the idea of "any codec", "any audio codec", and "any video codec". however, i'd recommend using audio/* and video/* instead of a* and v*, to avoid any potential confusion or surprise for future generations who might think to interpret e.g. a* to be like a shell glob/regex (that is, matching any fourCC starting with "a"), especially if * is the "any codec" wildcard. consider that there's already av01 (would match glob a*) and vp09 (would match glob v*), both video types, in the repertoire.

the situation i described up top, "the client supports receiving a codec i've never heard of, but i wish i knew if it was an audio codec i've never heard of or a video codec i've never heard of" is likely not going to be enough of a practical/functional concern to warrant significantly increasing the semantic complexity of encoding "here are the codecs i can receive". it might be of moderate interest for analytics, but even then i don't think we should increase the complexity of the everyday signaling just for that.

veovera commented 4 months ago

Fixed, please see https://veovera.org/docs/enhanced/enhanced-rtmp-v2.