Asserting decoding efficiency

[raphaelventura]

Hi,

Thanks for your work on the driver.

I set up the environment + kernel module options as stated and I think I managed to make it work on my machine: the logs look good, some compute info in nvtop / nvidia-smi. But unfortunately, I don't see much difference when playing high-resolution videos on youtube. I often get screen tearing, parts of the image going from "low-res" to "hi-res", and motion is not very smooth.

Here's my setup : Manjaro 23.1.2 Graphics card : Quadro K620 driver: linux61-nvidia 545.29.06-9

Also, I did not set gfx.x11-egl.force-enabled to true, since I use the direct backend (and I think that in turn, firefox should not use egl, am I right to think so?).

Excerpts from logs copied from stdout:

[RDD 3980: MediaPDecoder #2]: D/FFmpegVideo FFMPEG: Initialising VA-API FFmpeg decoder
[RDD 3980: MediaPDecoder #2]: D/FFmpegVideo FFMPEG:   Format h264 is accelerated
[RDD 3980: MediaPDecoder #2]: D/FFmpegVideo FFMPEG:   codec h264 : H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[RDD 3980: MediaPDecoder #2]: D/FFmpegVideo FFMPEG:   VA-API FFmpeg init successful
...
[RDD 3980: MediaPDecoder #2]: D/FFmpegVideo FFMPEG: Frame decode takes 0.14 ms average decode time 0.26 ms frame duration 16.67 average frame duration 16.67 decoded 5128 frames
[RDD 3980: MediaPDecoder #2]: D/FFmpegVideo FFMPEG: VA-API Got one frame output with pts=548633333 dts=548633333 duration=16666 opaque=-9223372036854775808

[AVHWDeviceContext @ 0x7f9ea2aa33c0] Format 0x3231564e -> nv12.
[AVHWDeviceContext @ 0x7f9ea2aa33c0] Format 0x30313050 -> p010le.
[AVHWDeviceContext @ 0x7f9ea2aa33c0] Format 0x32313050 -> p012le.
[AVHWDeviceContext @ 0x7f9ea2aa33c0] Format 0x36313050 -> unknown.
[AVHWDeviceContext @ 0x7f9ea2aa33c0] Format 0x50343434 -> yuv444p.
[AVHWDeviceContext @ 0x7f9ea2aa33c0] Format 0x36313451 -> unknown.
[AVHWDeviceContext @ 0x7f9ea2aa33c0] VAAPI driver: VA-API NVDEC driver [direct backend].
[AVHWDeviceContext @ 0x7f9ea2aa33c0] Driver not found in known nonstandard list, using standard behaviour.

[h264 @ 0x7f9ea2ae0000] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7f9ea2ae0000] Decoding VUI
[h264 @ 0x7f9ea2ae0000] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x7f9ea2ae0000] nal_unit_type: 5(IDR), nal_ref_idc: 1
[h264 @ 0x7f9ea2ae0000] Format vaapi chosen by get_format().
[h264 @ 0x7f9ea2ae0000] Format vaapi requires hwaccel initialisation.
[h264 @ 0x7f9ea2ae0000] Considering format 0x3231564e -> nv12.
[h264 @ 0x7f9ea2ae0000] Considering format 0x30313050 -> p010le.
[h264 @ 0x7f9ea2ae0000] Considering format 0x32313050 -> p012le.
[h264 @ 0x7f9ea2ae0000] Ignoring unknown format 0x36313050.
[h264 @ 0x7f9ea2ae0000] Picked nv12 (0x3231564e) as best match for yuv420p.

1172.705013393 [3980-5160] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:1300             nvEndPicture Decoded frame successfully to idx: 13 (0x7f9e87da5fc0)
1172.705026411 [3980-5229] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 391          resolveSurfaces Reading from queue: 7 8
1172.705052470 [3980-5229] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 407          resolveSurfaces Mapping surface 13
1172.705111353 [3980-5160] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2024    nvExportSurfaceHandle Exporting surface: 4 (0x7f9e87da53e0)
1172.705119227 [3980-5160] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c:2037    nvExportSurfaceHandle Exporting with 1920 1088 0 1920 3000000004fe014 0 1920 3 000000004fe014
1172.707942478 [3980-5229] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 411          resolveSurfaces Mapped surface 13 to 0x705e40000 (2048)
1172.707985541 [3980-5229] ../nvidia-vaapi-driver-0.0.11/src/vabackend.c: 415          resolveSurfaces Surface 13 exported

Note that my card only supports H264, and that I use firefox's h264ify extension.

Is there a way I could quantitatively assert that this indeed works ? I get the impression that reproducibility given some conditions (va-api used vs not used, playing the same video) in terms of "visual feeling" is not good.

Thanks

[elFarto]

The easiest way to see if the card is being used for decoding is to use nvidia-settings, in the 'GPU 0' section there is a 'Video Engine Utilization:' line which should be > 0% when in use.

You can also use nvidia-smi pmon, which will show you how much decoding each process is using.

[raphaelventura]

Thanks. I can indeed see some positive values for 'Video Engine Utilization' whenever logs print nice things.

I think my non-consistent experience corresponds to hardware acceleration not systematically working though. I'll make some more tests, but it also happens that, upon re-opening firefox after a successful decoding session, the new session doesn't handle it, and I get EGL-related errors.

Could you provide some insight about the value I should set this variable to ?

Also, I did not set gfx.x11-egl.force-enabled to true, since I use the direct backend (and I think that in turn, firefox should not use egl, am I right to think so?).

I checked that vainfo prints the supported profiles, and that my PC did not went to sleep in order to prevent the "post-sleep" bug mentioned in multiple issues.

[elFarto]

Firefox always uses EGL, it's how it imports the images this driver exports. The direct/egl backend switch changes how this driver exports those images, either via the proper 'supported' EGL method, or by calling the NVIDIA kernel driver directly (which is very unsupported). However we can't use the EGL method at the moment as it's been broken for over a year.

[raphaelventura]

Thanks for the clarification. I set the flag back to true.

I see that firefox sometimes issues error message about using hardware webrender at startup, which would explain the inconsistencies. I will close this issue and open another one concerning this bug.