search

fzwoch / obs-vaapi

OBS Studio VAAPI support via GStreamer
GNU General Public License v2.0
118 stars 3 forks source link

Encoding overloaded no matter what settings I use for 1080p 60fps #16

Closed T-Hitoshi closed 12 months ago

T-Hitoshi commented 12 months ago

Hello, so I came over from obs-gstreamer after a recommendation to switch to this instead. Previously I have been using obs-gstreamer because it is what works for me in 1080p 60fps. I have tried obs-vaapi before but kept encountering into "Encoding overloaded!" issues that I have not figured out a way to remedy since, but as mentioned works without issue when using the former.

I was looking into the list of open and closed issues and discovered other reports that mentions HEVC/H265, but on my end the overloading can also occur on H264.

Here are the details as I can provide:

inxi -SMC

System:
  Host: honoo-pc Kernel: 6.4.12-1-default arch: x86_64 bits: 64
    Desktop: KDE Plasma v: 5.27.7 Distro: openSUSE Tumbleweed 20230904
Machine:
  Type: Desktop System: ASUS product: N/A v: N/A serial: <superuser required>
  Mobo: ASUSTeK model: PRIME B560M-K v: Rev 1.xx
    serial: <superuser required> UEFI: American Megatrends v: 1605
    date: 05/13/2022
CPU:
  Info: 6-core model: 11th Gen Intel Core i5-11400F bits: 64 type: MT MCP
    cache: L2: 3 MiB
  Speed (MHz): avg: 2600 min/max: 800/2600 cores: 1: 2600 2: 2600 3: 2600
    4: 2600 5: 2600 6: 2600 7: 2600 8: 2600 9: 2600 10: 2600 11: 2600 12: 2600

inxi -Ga

Graphics:
  Device-1: AMD Polaris 20 XL [Radeon RX 580 2048SP] vendor: Sapphire
    driver: amdgpu v: kernel arch: GCN-4 code: Arctic Islands process: GF 14nm
    built: 2016-20 pcie: gen: 3 speed: 8 GT/s lanes: 16 ports:
    active: HDMI-A-1,HDMI-A-2 empty: DP-1,DP-2,DVI-D-1 bus-ID: 01:00.0
    chip-ID: 1002:6fdf class-ID: 0300 temp: 84.0 C
  Display: wayland server: X.org v: 1.21.1.8 with: Xwayland v: 23.2.0
    compositor: kwin_wayland driver: X: loaded: modesetting unloaded: fbdev,vesa
    dri: radeonsi gpu: amdgpu d-rect: 1920x2160 display-ID: 0
  Monitor-1: HDMI-A-1 pos: primary,top res: 1920x1080 size: N/A modes: N/A
  Monitor-2: HDMI-A-2 pos: bottom res: 1920x1080 size: N/A modes: N/A
  API: OpenGL v: 4.6 Mesa 23.1.6 renderer: AMD Radeon RX 580 2048SP
    (polaris10 LLVM 16.0.6 DRM 3.52 6.4.12-1-default) direct-render: Yes

vainfo

libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.19 (libva 2.19.0)
vainfo: Driver version: Mesa Gallium driver 23.1.6 for AMD Radeon RX 580 2048SP (polaris10, LLVM 16.0.6, DRM 3.52, 6.4.12-1-default)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

Logfile: 2023-09-08 12-20-45.txt

fzwoch commented 12 months ago

As on the other project's list, obs-gstreamer plugin does so at the expense of buffering all images.

With h.264 and h.265 and new and old implementations this plugin should offer 4 encoders. Check if they all the behave the same? Perhaps they differ at least by the amount of buffering?

Most of these issue will be connect to which Mesa version is being used and/or GStreamer runtime.

There is probably little this plugin can do as it is merely a bridge into OBS. The magic is done by other libraries.

As seen in https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1991 there still may be quirks.

fzwoch commented 12 months ago

Your first recording shows a pretty good result in my opinion:

2:21:20.638: ==== Recording Start ===============================================
12:21:20.638: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 12-21-20.mkv'...
12:22:04.576: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 12-21-20.mkv' stopped
12:22:04.576: Output 'adv_file_output': stopping
12:22:04.576: Output 'adv_file_output': Total frames output: 2630
12:22:04.576: Output 'adv_file_output': Total drawn frames: 2636
12:22:04.576: Video stopped, number of skipped frames due to encoding lag: 1/2635 (0.0%)
12:22:04.576: ==== Recording Stop ================================================

Not sure what else your are doing, but if you have a game or app running that uses your GPU non-throttled you probably take away required sources from the GPU it needs for encoding.

T-Hitoshi commented 12 months ago

Your first recording shows a pretty good result in my opinion:

2:21:20.638: ==== Recording Start ===============================================
12:21:20.638: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 12-21-20.mkv'...
12:22:04.576: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 12-21-20.mkv' stopped
12:22:04.576: Output 'adv_file_output': stopping
12:22:04.576: Output 'adv_file_output': Total frames output: 2630
12:22:04.576: Output 'adv_file_output': Total drawn frames: 2636
12:22:04.576: Video stopped, number of skipped frames due to encoding lag: 1/2635 (0.0%)
12:22:04.576: ==== Recording Stop ================================================

Not sure what else your are doing, but if you have a game or app running that uses your GPU non-throttled you probably take away required sources from the GPU it needs for encoding.

If I recall correctly the encoder is also overloaded in this one.

At other times, the recording/streaming starts looking okay at first but over time starts flashing the encoder overloaded warnings as recording goes on. This manifests in lags visible in the outputs (stream/recording).

Should be noted that my recording targets are the games I'm playing.

On an additional note, OBS-GStreamer on my hardware is capable of streaming at (scaled display, non-native) 1440p... in 30fps with barely any hiccups apart from connection lags on my end, and I think the end result looks very close to what I'm trying to accomplish. Would be nice if I can bump that up to 60fps, but as I've seen in the discussions for both plugins (obs-gstreamer and obs-vaapi) I guess the hardware still proves to be a bottleneck on that end.

T-Hitoshi commented 12 months ago

As on the other project's list, obs-gstreamer plugin does so at the expense of buffering all images.

With h.264 and h.265 and new and old implementations this plugin should offer 4 encoders. Check if they all the behave the same? Perhaps they differ at least by the amount of buffering?

Most of these issue will be connect to which Mesa version is being used and/or GStreamer runtime.

There is probably little this plugin can do as it is merely a bridge into OBS. The magic is done by other libraries.

As seen in https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1991 there still may be quirks.

I have tested all 4 encoders a few times, while the legacy ones seem to show a bit more stability they also succumb to overloading. On Flatpak the new ones do not show up for some reason, hence I had to fall back to system native OBS to test them.

Mesa on my system is at 23.1.6, while the gst-inspect-1.0 va output indicates it is at version 1.22.5, while the flatpak version is at 1.20.7. I am not sure if this helps, or if there exists any difference in determining GStreamer capabilities on my end.

fzwoch commented 12 months ago

If it works without the game it is indeed overloading the GPU. Not sure how many FPS you give to the game, but it feels it is not anything that can be fixed in this plugin.

T-Hitoshi commented 12 months ago

If it works without the game it is indeed overloading the GPU. Not sure how many FPS you give to the game, but it feels it is not anything that can be fixed in this plugin.

1080p 60fps.

Should have also mentioned that it still gets overloaded even when the target program is not running, i.e. showing a blank OBS canvas.

T-Hitoshi commented 12 months ago

Some fresh logs from a few recent recording attempts:

Recording at 1080p 60fps, H264-VAAPI: Overloaded.

info: [obs-vaapi] qos: 0
info: [obs-vaapi] min-force-key-unit-interval: 0
info: [obs-vaapi] key-int-max: 0
info: [obs-vaapi] b-frames: 0
info: [obs-vaapi] i-frames: 0
info: [obs-vaapi] ref-frames: 3
info: [obs-vaapi] b-pyramid: 0
info: [obs-vaapi] num-slices: 1
info: [obs-vaapi] min-qp: 1
info: [obs-vaapi] max-qp: 51
info: [obs-vaapi] qpi: 26
info: [obs-vaapi] qpp: 26
info: [obs-vaapi] qpb: 26
info: [obs-vaapi] dct8x8: 1
info: [obs-vaapi] cabac: 1
info: [obs-vaapi] trellis: 0
info: [obs-vaapi] mbbrc: disabled
info: [obs-vaapi] bitrate: 25
info: [obs-vaapi] target-percentage: 66
info: [obs-vaapi] target-usage: 7
info: [obs-vaapi] rate-control: cbr
info: [obs-vaapi] cpb-size: 0
info: [obs-vaapi] aud: 0
info: [obs-vaapi] cc-insert: 1
info: [obs-vaapi] codec: obs-va-vah264enc, 1920x1080@60/1, format: 2 
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 22-43-27.mkv'...
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 22-43-27.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 1277
info: Output 'adv_file_output': Total drawn frames: 3437 (3516 attempted)
info: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 79 (2.2%)
info: ==== Recording Stop ================================================
info: Video stopped, number of skipped frames due to encoding lag: 1550/1581 (98.0%)
info: libfdk_aac encoder destroyed

Recording at 1080p 60fps, HEVC-VAAPI: Overloaded.

info: [obs-vaapi] qos: 0
info: [obs-vaapi] min-force-key-unit-interval: 0
info: [obs-vaapi] key-int-max: 0
info: [obs-vaapi] b-frames: 0
info: [obs-vaapi] i-frames: 0
info: [obs-vaapi] ref-frames: 3
info: [obs-vaapi] b-pyramid: 0
info: [obs-vaapi] num-slices: 1
info: [obs-vaapi] min-qp: 1
info: [obs-vaapi] max-qp: 51
info: [obs-vaapi] qpi: 26
info: [obs-vaapi] qpp: 26
info: [obs-vaapi] qpb: 26
info: [obs-vaapi] trellis: 0
info: [obs-vaapi] mbbrc: disabled
info: [obs-vaapi] bitrate: 2500
info: [obs-vaapi] target-percentage: 66
info: [obs-vaapi] target-usage: 4
info: [obs-vaapi] rate-control: cbr
info: [obs-vaapi] cpb-size: 0
info: [obs-vaapi] aud: 0
info: [obs-vaapi] num-tile-cols: 1
info: [obs-vaapi] num-tile-rows: 1
info: [obs-vaapi] codec: obs-va-vah265enc, 1920x1080@60/1, format: 2 
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 22-48-47.mkv'...
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 22-48-47.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 1206
info: Output 'adv_file_output': Total drawn frames: 2918 (3033 attempted)
info: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 115 (3.8%)
info: ==== Recording Stop ================================================
info: Video stopped, number of skipped frames due to encoding lag: 1218/1251 (97.4%)
info: libfdk_aac encoder destroyed

Same resolution and framerate, H264-VAAPI (Legacy): Overloaded.

info: [obs-vaapi] qos: 0
info: [obs-vaapi] min-force-key-unit-interval: 0
info: [obs-vaapi] bitrate: 2500
info: [obs-vaapi] target-percentage: 70
info: [obs-vaapi] keyframe-period: 30
info: [obs-vaapi] quality-level: 4
info: [obs-vaapi] default-roi-delta-qp: -10
info: [obs-vaapi] trellis: 0
info: [obs-vaapi] rate-control: cbr
info: [obs-vaapi] tune: none
info: [obs-vaapi] max-bframes: 0
info: [obs-vaapi] init-qp: 26
info: [obs-vaapi] min-qp: 1
info: [obs-vaapi] num-slices: 1
info: [obs-vaapi] cabac: 0
info: [obs-vaapi] dct8x8: 0
info: [obs-vaapi] cpb-length: 1500
info: [obs-vaapi] num-views: 1
info: [obs-vaapi] aud: 0
info: [obs-vaapi] compliance-mode: strict
info: [obs-vaapi] refs: 1
info: [obs-vaapi] mbbrc: auto
info: [obs-vaapi] qp-ip: 0
info: [obs-vaapi] qp-ib: 0
info: [obs-vaapi] temporal-levels: 1
info: [obs-vaapi] prediction-type: default
info: [obs-vaapi] max-qp: 51
info: [obs-vaapi] quality-factor: 26
info: [obs-vaapi] codec: obs-vaapi-vaapih264enc, 1920x1080@60/1, format: 2 
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 22-51-14.mkv'...
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 22-51-14.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 916
info: Output 'adv_file_output': Total drawn frames: 1383 (1473 attempted)
info: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 90 (6.1%)
info: ==== Recording Stop ================================================
info: Video stopped, number of skipped frames due to encoding lag: 949/1002 (94.7%)
info: libfdk_aac encoder destroyed

Same resolution and framerate, HEVC-VAAPI (Legacy): Overloaded.

info: [obs-vaapi] device: 
info: [obs-vaapi] qos: 0
info: [obs-vaapi] min-force-key-unit-interval: 0
info: [obs-vaapi] bitrate: 2500
info: [obs-vaapi] target-percentage: 70
info: [obs-vaapi] keyframe-period: 30
info: [obs-vaapi] quality-level: 4
info: [obs-vaapi] default-roi-delta-qp: -10
info: [obs-vaapi] trellis: 0
info: [obs-vaapi] rate-control: cbr
info: [obs-vaapi] tune: none
info: [obs-vaapi] max-bframes: 0
info: [obs-vaapi] init-qp: 26
info: [obs-vaapi] min-qp: 1
info: [obs-vaapi] num-slices: 1
info: [obs-vaapi] refs: 1
info: [obs-vaapi] cpb-length: 1500
info: [obs-vaapi] mbbrc: auto
info: [obs-vaapi] qp-ip: 0
info: [obs-vaapi] qp-ib: 0
info: [obs-vaapi] low-delay-b: 0
info: [obs-vaapi] max-qp: 51
info: [obs-vaapi] quality-factor: 26
info: [obs-vaapi] num-tile-cols: 1
info: [obs-vaapi] num-tile-rows: 1
info: [obs-vaapi] codec: obs-vaapi-vaapih265enc, 1920x1080@60/1, format: 2 
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 22-53-58.mkv'...
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 22-53-58.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 1394
info: Output 'adv_file_output': Total drawn frames: 1816 (1941 attempted)
info: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 125 (6.4%)
info: ==== Recording Stop ================================================
info: Video stopped, number of skipped frames due to encoding lag: 1358/1436 (94.6%)
info: libfdk_aac encoder destroyed

Recording absolutely nothing (blank OBS canvas), H264-VAAPI:

info: [obs-vaapi] qos: 0
info: [obs-vaapi] min-force-key-unit-interval: 0
info: [obs-vaapi] key-int-max: 0
info: [obs-vaapi] b-frames: 0
info: [obs-vaapi] i-frames: 0
info: [obs-vaapi] ref-frames: 3
info: [obs-vaapi] b-pyramid: 0
info: [obs-vaapi] num-slices: 1
info: [obs-vaapi] min-qp: 1
info: [obs-vaapi] max-qp: 51
info: [obs-vaapi] qpi: 26
info: [obs-vaapi] qpp: 26
info: [obs-vaapi] qpb: 26
info: [obs-vaapi] dct8x8: 1
info: [obs-vaapi] cabac: 1
info: [obs-vaapi] trellis: 0
info: [obs-vaapi] mbbrc: disabled
info: [obs-vaapi] bitrate: 2500
info: [obs-vaapi] target-percentage: 66
info: [obs-vaapi] target-usage: 4
info: [obs-vaapi] rate-control: cbr
info: [obs-vaapi] cpb-size: 0
info: [obs-vaapi] aud: 0
info: [obs-vaapi] cc-insert: 1
info: [obs-vaapi] codec: obs-va-vah264enc, 1920x1080@60/1, format: 2 
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 22-58-11.mkv'...
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 22-58-11.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 3335
info: Output 'adv_file_output': Total drawn frames: 3337 (3339 attempted)
info: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 2 (0.1%)
info: Video stopped, number of skipped frames due to encoding lag: 1/3339 (0.0%)
info: ==== Recording Stop ================================================
info: libfdk_aac encoder destroyed

Recording OBS terminal output, H264-VAAPI:

info: User switched to scene 'draw(xshm)'
info: User switched to scene 'game2(xshm cap)'
info: [window-capture: 'Window Capture (Xcomposite)'] update settings:
        title: obs
        class: xterm

info: [obs-vaapi] qos: 0
info: [obs-vaapi] min-force-key-unit-interval: 0
info: [obs-vaapi] key-int-max: 0
info: [obs-vaapi] b-frames: 0
info: [obs-vaapi] i-frames: 0
info: [obs-vaapi] ref-frames: 3
info: [obs-vaapi] b-pyramid: 0
info: [obs-vaapi] num-slices: 1
info: [obs-vaapi] min-qp: 1
info: [obs-vaapi] max-qp: 51
info: [obs-vaapi] qpi: 26
info: [obs-vaapi] qpp: 26
info: [obs-vaapi] qpb: 26
info: [obs-vaapi] dct8x8: 1
info: [obs-vaapi] cabac: 1
info: [obs-vaapi] trellis: 0
info: [obs-vaapi] mbbrc: disabled
info: [obs-vaapi] bitrate: 2500
info: [obs-vaapi] target-percentage: 66
info: [obs-vaapi] target-usage: 4
info: [obs-vaapi] rate-control: cbr
info: [obs-vaapi] cpb-size: 0
info: [obs-vaapi] aud: 0
info: [obs-vaapi] cc-insert: 1
info: [obs-vaapi] codec: obs-va-vah264enc, 1920x1080@60/1, format: 2 
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/honoo/Videos/2023-09-08 23-00-16.mkv'...
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/honoo/Videos/2023-09-08 23-00-16.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 3190
info: Output 'adv_file_output': Total drawn frames: 3195 (3197 attempted)
info: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 2 (0.1%)
info: Video stopped, number of skipped frames due to encoding lag: 1/3196 (0.0%)
info: ==== Recording Stop ================================================
info: libfdk_aac encoder destroyed
DonKatsu commented 11 months ago

I'm not sure where else to bring this up so I'll do it here.

I tried Mechwarrior Online with MSAA today, and was recording as I loaded into Forest Colony. Everything was normal until I got in to the main wooded area, where my GPU utilization actually maxed out. In OBS time to render frame jumped to 20ms+, FPS dropped to the 40s, and missed/skipped were piling up. Switched encoder to obs-vaapi h265 legacy, and even ffmpeg vaapi hevc, same behaviors. Turned off MSAA and restarted MWO. Loaded up that map in training and GPU usage was back down to barely anything, and no encoder had any issue again.

None of the vaapi encoders show any odd behavior while in use with Deep Rock Galactic or Armored Core 6 hitting 100% GPU utilization, so I'm stumped.