search

AOMediaCodec / av1-rtp-spec

Current draft (HTML): https://aomediacodec.github.io/av1-rtp-spec/
Other
63 stars 24 forks source link

Definition of Instantaneous Decidability of Decodability (IDD) #215

Closed aboba closed 3 years ago

aboba commented 3 years ago

IDD isn't defined in the terminology section, only in Section A.6:

"Chains provide the IDD property. That is, the ability to decide immediately upon receiving the very first packet after packet loss, if any of the lost packets are required for the Decode target to remain decodable."

This definition mixes loss recovery and decodability issues. When an endpoint receives a packet with a sequence number gap, this could be due to re-ordering rather than loss, and FEC could permit recovery of the missing packet without retransmission. However, the mechanics of loss recovery isn't really relevant because IDD is primarily about the information that an SFM needs to make the decision whether to immediately forward a packet to one or more endpoints.

aboba commented 3 years ago

Here is a definition from the SFM perspective:

"IDD: The ability of an SFM to decide immediately on receiving a packet whether to forward it to one or more downstream endpoints. This requires the SFM to be able to determine whether:
1) The packet belongs to a frame which is required for the endpoint's Decode target. This is provided by the Decode Target Indication (DTI) values. 2) All frames referenced by the current frame have been forwarded to the endpoint. This is provided by the frame difference values (see Section A8.3). 3) All frames referenced by the current frame are decodable. This is provided by Chain information (see Section A.5) which is present in all packets, allowing an SFM to detect a broken Chain regardless of loss patterns."

DanilChapovalov commented 3 years ago

2 is achieved with frame diff field but is not sufficient to decide if packet can be forwarded. Example A.10.1.1 describes scenario where frame F5 has all frames referenced been forwarded, but still should not yet be forwarded in some scenarios. That's where chains are helpful.

aboba commented 3 years ago

Right. See revised definition above.

DanilChapovalov commented 3 years ago

It still has the same problem. 3 can be achieved by looking at frame difference values and remembering which forwarded frames have their dependencies fulfilled (or by avoiding forwarding frames with unfulfilled dependencies) Chain are there to tell if missed packets/frames are essential for other, future frames, regardless if currently received frame is decodable by itself.

I think there are two kind of SFMs: one kind preserve rtp sequence number and tries to forward packets/ frames asap, sometimes leaving gaps in decodability. Another kind forwards only frames that are decodable. Such SFM can override rtp sequence number to be consecutive even if some layers are not forwarded. Which SFM do you have in mind? First kind probably doesn't need chains (and IDD as I understand it)

IDD/Chains can also be relied on by decoder/final endpoint. Example A.10.1.1 describes usefulness of chains from such decoder point of view. Chains prevent situation where S0 frame from 2nd temporal unit is decoded before S1 frame from 1st temporal unit. Both such frames depend just on S0 frame from 1st temporal unit, but they have to be decoded in correct order.

StephenBotzko commented 3 years ago

I suggest the following adjustment to the preamble of the definition:

"IDD: The ability to decide immediately whether a packet is needed to decode a received bitstream. This property allows an SFM to determine whether to forward a packet to one or more downstream endpoints. This property is satisfied when the receiver is able to determine whether:"

Also, I believe that we can continue to call the property IDD. We can simply adjust the text in A.6 to say "Chains provide an important component of the IDD property"

aboba commented 3 years ago

@StephenBotzko Looks good to me.

DanilChapovalov commented 3 years ago

The property provided by chains is not that much about currently received packet, but mostly about missed packets before the currently received packet. Chains provide ability to decide where a missing packet is needed to decode the received bitstream. Looking at the first sentence in the new definition it might be too subtle to notice that "a packet" can mean both "a missed" and "a received" packet. Moreover, when talking about "missed" packet it becomes unclear what "immediately" means.

If missed packets weren't meant, then that property is not strong enough, in particular chains do not provide a component for it.

StephenBotzko commented 3 years ago
IDD: The ability to decide immediately whether a received or missed packet is needed to decode a received bitstream. This property allows an SFM to determine whether to forward a packet to one or more downstream endpoints or attempt to recover essential missed packets. The property is satisfied when the receiver is able to determine whether: 1: The packet belongs to a frame which is required for the endpoint's Decode target. This is provided by the Decode Target Indication (DTI) values. 2: All frames referenced by the current frame have been forwarded to the endpoint. This is provided by the frame difference values (see Section A8.3). 3: Upon receiving the very first packet after an RTP sequence number gap, if any of the missed packets are required for the Decode target to remain decodable. This is provided by the chain information (see Section A5). Add a note someplace else that the template needs to be received?
StephenBotzko commented 3 years ago

Closed with PR #222