This PR unfortunately doesn't work and I ran out of ideas on how to push it further. Current status:
chromium successfully decodes and displays the first video frame
I can't see any logs indicating we are doing something wrong. I ran dev build of chromium. The only thing I can see is
[2098991:3:1211/162436.394069:WARNING:dav1d_decoder.cc(135)] Dav1dDecoder::Decode decoding failed with error code -11
[2098991:3:1211/162436.394213:WARNING:generic_decoder.cc(302)] Failed to decode frame with timestamp 2618223154, error code: -1
According to dav1d IIRC this might not be the error but I think it shouldn't be logged
I assumed that sending every OBU in a separate RTP packet is ok
I assumed that every frame is a separate Temporal Unit (in the test file this should be true if only the code checking OBU type is correct) and therefore I set marker of the last RTP packet for every IVF frame
I tried batching OBUs e.g. by 2 or setting W flag to 1 (instead of 0 and using LEB size) - didn't help
I tried prepending every OBU with sequence header - this gives a massive number of decoder errors (-11)
Pion send_from_file example also doesn't work but here I am getting
[2077476:12:1211/123520.805336:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 591
[2077476:12:1211/123520.838810:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.871969:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 33
[2077476:12:1211/123520.904092:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.936935:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 62
[2077476:12:1211/123520.970613:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123520.996106:WARNING:video_receive_stream2.cc(825)] No decodable frame in 230195 us, requesting keyframe.
[2077476:12:1211/123521.003523:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 33
[2077476:12:1211/123521.037086:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.069772:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 180
[2077476:12:1211/123521.102389:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.135413:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 28
[2077476:12:1211/123521.168366:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 3
[2077476:12:1211/123521.201628:WARNING:video_rtp_depacketizer_av1.cc(314)] Mismatch in obu_size. signaled: 0, actual: 95
[2077476:12:1211/123521.214954:WARNING:video_receive_stream2.cc(825)] No decodable frame in 218582 us, requesting keyframe.
if I don't drop Temporal Units, I can see
[2080197:11:1211/130520.654168:WARNING:rtp_seq_num_only_ref_finder.cc(65)] Generic frame with packet range [51605, 51605] has no GoP, dropping frame.
maybe this has something to do with this paragraph of AV1 RTP format specification (see here)
If a sequence header OBU is present in an RTP packet and operating_points_cnt_minus_1 > 0 then for any number i where 0 <= i < operating_points_cnt_minus_1 the following MUST be true: (operating_point_idc[i] & operating_point_idc[i+1]) == operating_point_idc[i+1].
A sender MAY produce a sequence header with operating_points_cnt_minus_1 = 0 and operating_point_idc[0] = 0xFFF and seq_level_idx[0] = 0. In such case, seq_level_idx[0] does not reflect the level of the operating point.
Note: The intent is to disable OBU dropping in the AV1 decoder. To ensure a decoder’s capabilities are not exceeded, OBU filtering should instead be implemented at the system level (e.g., in a MANE).
but because it sounds pretty complex I didn't dive into it
This PR unfortunately doesn't work and I ran out of ideas on how to push it further. Current status:
According to dav1d IIRC this might not be the error but I think it shouldn't be logged
send_from_fileexample also doesn't work but here I am gettingA sender MAY produce a sequence header with operating_points_cnt_minus_1 = 0 and operating_point_idc[0] = 0xFFF and seq_level_idx[0] = 0. In such case, seq_level_idx[0] does not reflect the level of the operating point.
Note: The intent is to disable OBU dropping in the AV1 decoder. To ensure a decoder’s capabilities are not exceeded, OBU filtering should instead be implemented at the system level (e.g., in a MANE).