ehfd
commented
1 year ago #42 implements Moreover, performance with vaapih264enc. However, for Intel GPUs this is only limited to LIBVA_DRIVER_NAME=i965, which means that the iHD driver is unable to be used.ximagesrc is very low with Xorg on Intel GPUs somehow.
vah264enc is going to come in GStreamer 1.21 or 1.22. This is a more modern implementation, and vaapih264enc will get frozen. Investigate this when it comes.i965 and iHD.
AMD's AMF API and Intel's Media SDK may also deliver better performance. Therefore, this issue is not being closed and more contribution is needed.
Note: https://blogs.igalia.com/vjaquez/2018/03/28/gstreamer-va-api-troubleshooting/
Relevant code: https://github.com/selkies-project/selkies-gstreamer/blob/master/src/selkies_gstreamer/gstwebrtc_app.py Anyone with GStreamer experience, or willing to learn can solve this.
Supporting GPU statistics is also needed. GPUtils only supports NVIDIA.
maxpain
Looking for contributors to help out.
Guide for improving video encoder parameters (also applicable to non-NVIDIA encoders): https://docs.nvidia.com/video-technologies/video-codec-sdk/12.2/nvenc-video-encoder-api-prog-guide/index.html#recommended-nvenc-settings__table-nvenc-settings
Remember that live (as in < 100 ms) streaming is all about keeping the bitrate consistent at all frames to minimize congestion from unexpected bandwidth spikes (which lead to stutters and latency spikes) and keeping the buffer small. Use the configured Constant Bitrate bandwidth as possible at all times but do not go over it (ideal maximum overshoot tolerance = 10%).
Priority: Latency > Quality (minimum sacrifices of <= 5% in latency for quality or bandwidth consistency is acceptable, using more GPU resources or CPU cores also acceptable if without latency impact) >> Overall Bandwidth Efficiency (since encoded frames are only used once unlike .mp4/.mkv)
This is also applicable to other encoders if available.
*: These features are useful for error recovery during transmission across noisy mediums. For usecases where the client requires reduced video memory footprint (this typically means reducing any buffer that exists, preventing frame lag, except the ones unrelated to latency, stated below**), following guidelines should be followed.