paullouisageneau / libdatachannel

C/C++ WebRTC network library featuring Data Channels, Media Transport, and WebSockets
https://libdatachannel.org/
Mozilla Public License 2.0
1.67k stars 343 forks source link

VP8 packetizer #558

Open paullouisageneau opened 2 years ago

paullouisageneau commented 2 years ago

VP8 packetization should be added similarly to H264 packetization.

The RTP payload format is described in RFC 7741.

Wojtab commented 2 years ago

I was implementing it one time, wanted to contribute it, but didn't have time to finish. Here is a very limited and terrible implementation, which expects single-frame samples(I was grabbing them from ffmpeg with libvpx using avcodec_send_frame and then reading with avcodec_receive_packet until no data was available). It was working in the streamer example with Chrome and some Unity plugin, just need to modify the addVideo with the following:

#include "vp8rtp.h"
std::shared_ptr<ClientTrackData> addVP8Video(const std::shared_ptr<rtc::PeerConnection> pc, const uint8_t payloadType, const uint32_t ssrc, const std::string cname, const std::string msid, const std::function<void ()> onOpen) {
    auto video = rtc::Description::Video(cname);
    video.addVP8Codec(payloadType);
    video.addSSRC(ssrc, cname, msid, cname);
    auto track = pc->addTrack(video);
    // create RTP configuration
    auto rtpConfig = std::make_shared<rtc::RtpPacketizationConfig>(ssrc, cname, payloadType, rtc::H264RtpPacketizer::defaultClockRate);
    // create packetizer
    auto packetizer = std::make_shared<VP8Rtp>(rtpConfig);
    // create H264 handler
    auto vp8Handler = std::make_shared<VP8RtpHandler>(packetizer);
    // add RTCP SR handler
    auto srReporter = std::make_shared<rtc::RtcpSrReporter>(rtpConfig);
    vp8Handler->addToChain(srReporter);
    // add RTCP NACK handler
    auto nackResponder = std::make_shared<rtc::RtcpNackResponder>();
    vp8Handler->addToChain(nackResponder);
    // set handler
    track->setMediaHandler(vp8Handler);
    track->onOpen(onOpen);
    auto trackData = std::make_shared<ClientTrackData>(track, srReporter);
    return trackData;
}

By limited I mean it doesn't support any extensions like PictureId, but the basic I-Frame/P-frame packetization works vp8rtp.zip

paullouisageneau commented 2 years ago

@Wojtab Thank you, that looks like a good base!

jacobsologub commented 8 months ago

@Wojtab Thanks for sharing this! Plugged this in and it works like a charm. What would need to change for vp9?

chobie commented 8 months ago

@jacobsologub regarding VP9 packetizer, it's not using the current MediaHandler's API, but I think the next commit could be a reference.

https://github.com/paullouisageneau/libdatachannel/compare/master...chobie:libdatachannel:vp8_vp9#diff-89f993916bb1675b6754adf38ed6a63a5ed9a921589e98a10948ebce4e5c9335R18 this commit only implements 2 byte descriptor. but It was working as far as I tested with VP9 encoding using libvpx locally.

Please refer to the following RFC for details: https://datatracker.ietf.org/doc/html/draft-ietf-payload-vp9-16

Considering depacketization, it will be necessary to rewrite. so I am currently prioritizing other tasks.

jacobsologub commented 8 months ago

@chobie just checked out your fork, worked right out of the box.