ietf-wg-avtcore / draft-ietf-avtcore-hevc-webrtc

Other
5 stars 2 forks source link

NALU TID values #25

Open aboba opened 1 week ago

aboba commented 1 week ago

The specification doesn't currently talk about packetization constraints on TID values in NAL units. For example, RFC 7798 Section 4.4.2 states:

"The value of TID MUST be the lowest value of TID of all the aggregated NAL units.

Informative note: All VCL NAL units in an AP have the same TID value since they belong to the same access unit. However, an AP may contain non-VCL NAL units for which the TID value in the NAL unit header may be different than the TID value of the VCL NAL units in the same AP."

However, Including NAL units with distinct TIDs in an RTP payload is problematic, because SFMs that use RTP header extensions for forwarding typically support only a single TID value. If the lowest TID value is used in the RTP header extension, then a receiver can receive a payload including NAL units for a TID at a higher operating point than that chosen by the SFM (e.g. SFM is only sending base layer frames to a participant, but the payload might also include an extension layer non-VCL NAL unit).

Here is a proposal:

taste1981 commented 1 week ago

In Chromium implementation we expect dependency descriptor (DD) to be setup for HEVC. A DD-aware SFM should rely on DD instead of checking AP payload header for forwarding.

With that said, I don't see in RFC 7798 that PayloadHdr must be not encrypted, though in practice I believe it should be clear.

aboba commented 1 week ago

This is an area that is not well documented for any codec. For use in EncodedTransform API, the mime-type is not changed, so there are expectations that PayloadHdr (or at least elements of it) are in the clear.

This requirement can be relaxed if the SDP does not claim to be send/receiving video/H.265 but instead declares that it is using SFRAME.