Does the argument binary_ptr message of AV1RtpPacketizer::packetizeObu expect a single obu unit?

[chobie]

I tried to see if I could stream video using libaom. After aom_codec_encode, retrieve the encoded frame with aom_codec_get_cx_data. However, when I transmitted that frame, Chrome did not recognize it as a frame and it was dropped. Upon analyzing the encoded frame, I found that it apparently contains multiple obu units.

When I tried transmitting the frame after dividing it by obu units, the video was displayed as expected.

Do I understand correctly that the current packetizeObu expects a single obu unit?

here is pseudo code

error = aom_codec_encode(&context->codec, raw, frame_index, 60000, flags);
if (error != AOM_CODEC_OK) {
    return error_codec(&context->codec, "Failed to encode frame.\n");
}

while ((pkt = aom_codec_get_cx_data(&context->codec, &iter))) {
    got_pkts = 1;

    if (pkt->kind == AOM_CODEC_CX_FRAME_PKT) {
        const int keyframe = (pkt->data.frame.flags & AOM_FRAME_IS_KEY) != 0;

        // NOTE: pkt->data.frame.buf contains multiple obu untis.
        // so, we have to split frame data into obu unit. otherwise we can't packetize correctly with AV1RtpPacketizer::packetizeObu.
        // (I didn't show split code as i don't understand obu spec yet.)
        callback(obu_unit, obu_unit_size, pkt->data.frame.pts);
    }
}

[chobie]

While testing, I found a another issue and reporting it here. https://github.com/paullouisageneau/libdatachannel/blob/master/src/av1rtppacketizer.cpp#L132

It seems that when the sequence header is cached, and there is only one obu unit, the sequence header progresses without being consumed.

In such cases, an assertion fails and terminates at the following location: https://github.com/paullouisageneau/libdatachannel/blob/master/src/mediahandlerelement.cpp#L164

[chobie]

here is av1-sender patch (modified media-sender). it's not correct but it works roughly.

https://gist.github.com/chobie/49d219a6c658de3667231016b8c1d5e8

https://github.com/paullouisageneau/libdatachannel/assets/195038/7e475c96-3e90-4f8d-b976-ef459b48cb47 (this movie set keyframe interval is 1)

[paullouisageneau]

Great job, thank you for sharing!

While testing, I found a another issue and reporting it here. https://github.com/paullouisageneau/libdatachannel/blob/master/src/av1rtppacketizer.cpp#L132

It seems that when the sequence header is cached, and there is only one obu unit, the sequence header progresses without being consumed.

In such cases, an assertion fails and terminates at the following location: https://github.com/paullouisageneau/libdatachannel/blob/master/src/mediahandlerelement.cpp#L164

@Sean-Der Could you please look into this?