Glimesh / janus-ftl-plugin

A plugin for the Janus WebRTC gateway to enable relaying of audio/video streams utilizing Mixer's FTL (Faster-Than-Light) protocol.
https://hayden.fyi/posts/2020-08-03-Faster-Than-Light-protocol-engineering-notes.html
GNU Affero General Public License v3.0
44 stars 11 forks source link

VideoDecoder decodes keyframe to retrieve video resolution #120

Open haydenmc opened 3 years ago

haydenmc commented 3 years ago

As part of #87, VideoDecoder is now capable of retrieving the width/height of a video stream. However, to do this, it must be provided a keyframe sequence that it will decode via libav. As a result, the metadata report thread now ends up decoding keyframes twice - first to retrieve the video dimensions, and a second time to generate a JPEG preview image.

It's also worth considering using a different approach to extract the resolution from H264 video. SPS NAL units should contain the information we need. There's some good reference code from the WebRTC implementation here. This will save us from having to decode an entire frame just to get the width/height information.