Closed TheSashmo closed 1 year ago
Is it possible that 10Mbps video in quicksync is not good? I miss-typed a command and set it to 1Mbps.... And now there is not stutter... What I am missing here? Suggestions?
Have you tried forward error correction?
Its local host.... Shouldn't need any FEC. Also its not reporting any lost packets, so no need for FEC.
Ah sorry I assumed this was over the network... what kind of video artifacts are you seeing? About 7 months ago I was testing quicksync decode and saw intermittent corrupt video frames on the output. I was never able to finish testing though...
Just a skip in the video. Looks like the frame was dropped. No artifacts at all. It's not consistently smooth.
Hi @TheSashmo, I'll try to check it out, could you respond me few questions?
--param lavc-use-codec=vaapi_vld
, eg. it won't work without that explicitly?I may get some (or all) answers during the testing but it may be quite easy to answer.
Hi @MartinPulec
Answers to your questions:
00:02.0 VGA compatible controller: Intel Corporation Device 9bca (rev 04) (prog-if 00 [VGA controller]) DeviceName: Onboard - Video Subsystem: Intel Corporation Device 2212 Flags: bus master, fast devsel, latency 0, IRQ 131 Memory at b0000000 (64-bit, non-prefetchable) [size=16M] Memory at a0000000 (64-bit, prefetchable) [size=256M] I/O ports at 5000 [size=64] Expansion ROM at 000c0000 [virtual] [disabled] [size=128K] Capabilities: <access denied> Kernel driver in use: i915 Kernel modules: i915
Here is what I know works:
Here is what doesn't work:
I was attempting to do I and P encode decode on the same device, and the 1080p60 wont encode and decode using that quick sync configurations, but I can do both encode and decode 1080p60 on CPU only.
I can do Interlaced only with quick sync, and it works very well. I find that if I only use very low bitrate option the video is smooth and the quality is fine. If I set to 5Mbps or higher, I can see that the video gets choppy on the output, where it looses 1-2 frames in any motion. i.e. testing with my SDI camera and moving my head around or wheeling my chair across the floor, I can see the frame skip. If it's static talking head, it's almost not noticeable, but for anything else then that its not usable.
I've just noticed, that I've missed one thing - for the decoder, if you intend to use VA-API, there is not a standalone decoder hevc_vaapi in FFmpeg/libavcodec, you'd need to use HW acceleration instead - --param use-hw-accel
(see here).
The hevc_vaapi decoder doesn't AFAIK exist - I'll change that UG should perhaps report an error and exit in this situation. For a standalone option, you may try to use the "native" QuickSync decoder (--param decoder-use-codec=hevc_qsv
), but it may not be available in your setup (I guess that packages in U20.04 should support that), you could check it in the output of -c libavcodec:help
.
@MartinPulec Ok Just to understand what you are saying..
You are suggesting to use the --param user-hw-accel instead of using the command that I shared above --param force-lavd-decoder=hevc_vaapi ?
There is no hevc_qsv but there is hevc_vaapi from what I was reading that the libmfx and vaapi differences are minor. I did install the msdk but was only able to make it all work running the commands I shared in my example above.
The only main issue that I am still trying to identify is why when I can do Interlaced only with quick sync, and it works very well. I find that if I only use very low bitrate option the video is smooth and the quality is fine. If I set to 5Mbps or higher, I can see that the video gets choppy on the output, where it looses 1-2 frames in any motion. i.e. testing with my SDI camera and moving my head around or wheeling my chair across the floor, I can see the frame skip. If it's static talking head, it's almost not noticeable, but for anything else then that its not usable.
You are suggesting to use the --param user-hw-accel instead of using the command that I shared above --param force-lavd-decoder=hevc_vaapi ?
Yes, I think that --param force-lavd-decoder=hevc_vaapi
doesn't have any effect since there is no hevc_vaapi decoder (only the encoder). But you could check it in the output, eg. I get (Ubuntu 21.04):
uv -c libavcodec:help | grep hevc
H.265 - available (encoders: libx265 nvenc_hevc hevc_nvenc hevc_qsv hevc_v4l2m2m hevc_vaapi, decoders: hevc hevc_qsv hevc_v4l2m2m hevc_cuvid)
There is no hevc_qsv but there is hevc_vaapi from what I was reading that the libmfx and vaapi differences are minor. I did install the msdk but was only able to make it all work running the commands I shared in my example above.
Which OS do you use? FFmpeg in U20.04 should be built with QSV support. I've also tried to add the support to the AppImage, but it is problematic. But when using the packages, it only depends on what is present in system.
You are suggesting to use the --param user-hw-accel instead of using the command that I shared above --param force-lavd-decoder=hevc_vaapi ?
Yes, I think that
--param force-lavd-decoder=hevc_vaapi
doesn't have any effect since there is no _hevcvaapi decoder (only the encoder). But you could check it in the output, eg. I get (Ubuntu 21.04):uv -c libavcodec:help | grep hevc H.265 - available (encoders: libx265 nvenc_hevc hevc_nvenc hevc_qsv hevc_v4l2m2m hevc_vaapi, decoders: hevc hevc_qsv hevc_v4l2m2m hevc_cuvid)
There is no hevc_qsv but there is hevc_vaapi from what I was reading that the libmfx and vaapi differences are minor. I did install the msdk but was only able to make it all work running the commands I shared in my example above.
Which OS do you use? FFmpeg in U20.04 should be built with QSV support. I've also tried to add the support to the AppImage, but it is problematic. But when using the packages, it only depends on what is present in system.
All my testing is on Ubuntu 20.04. My output shows the same as yours.
I could just now be confusing how the encoding configuration for using QSV is being used.
Ok, I see, but in that case you'd really need to use --param use-hw-accel
to use VA-API decoding. If I am right, specifying non-existent decoder would now fail with current AppImage.
Thanks I will attempt that.
So bringing this back to life without opening another issue. I am trying to achieve two HDMI outputs on one machine. The device is a Intel(R) Celeron(R) J4125 CPU @ 2.00GHz with two HDMI ports on it. I can easily play one output without any issue. When I attempt to play two signals on it, I can clearly see a dropped frame rate in the decoding where from 150 frames it drops occasionally to 148 or less frames. So I am attempting to use the --param use-hw-accel option, and I get the following error: `[h264 @ 0x7f0f08045c00] no frame! libva info: VA-API version 1.1.0es libva info: va_getDriverName() returns 0 libva info: Trying to open /tmp/.mount_UltraGNuP5Mz/usr/lib/va/iris_drv_video.so libva info: va_openDriver() returns -1
[lavd] Vdpau copy mode enabled because the decoder wasn't configured to output HW_VDPAU (maybe the display doesn't support it) This may be slower than sw decoding. [h264 @ 0x7f0f08045c00] Failed setup for format vdpau: hwaccel initialisation returned error.`
Here is the decode options in -c libavcodec:help
H.264 - available (encoders: libx264 libx264rgb h264_nvenc h264_v4l2m2m h264_vaapi, decoders: h264 h264_crystalhd h264_v4l2m2m h264_cuvid)
I dont see the vaapi decode available, but I have seen it for sure on other machines.
The funny thing is I can run kodi on the same machine and do dual 4k signals (downscaled to 1080) on the same machine without going over 40% cpu or 1.9 load average on that 4 core system.
So the hardware and the OS have the abilityf, but I am not able to get it to work correctly with my setup.
Here is what kodi shows at runtime
libva info: VA-API version 1.13.0 libva info: Trying to open /usr/local/lib/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_13 libva info: va_openDriver() return
Ffmpeg shows that only these decoders are available, as what ultragrid reports
VFS..D h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 V..... h264_v4l2m2m V4L2 mem2mem H.264 decoder wrapper (codec h264) V..... h264_cuvid Nvidia CUVID H264 decoder (codec h264)
Even though cuvid is there, there is no nvidia card on this system.
I have tried all the variants of the h264 decoders available. I've even tried to install ffmpeg with vaapi (from apt) but that dosnt seem to help either. I will attempt compiling again by hand and see if that makes any difference in seeing the vaapi decoder.
Here is the information from vainfo:
libva info: VA-API version 1.13.0 libva info: Trying to open /usr/local/lib/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_13 libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.13 (libva 2.13.0.pre1) vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.3.1 (36e99728) vainfo: Supported profile and entrypoints VAProfileNone : VAEntrypointVideoProc VAProfileNone : VAEntrypointStats VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointFEI VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264High : VAEntrypointFEI VAProfileH264High : VAEntrypointEncSliceLP VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointFEI VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointFEI VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSlice VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD
Suggestions?
Update I tried to recompile ffmpeg as well, and same result.
Hi, while the log you posted contains some libva output, it looks like UltraGrid is trying to initialize vdpau here (as shown by the "Failed setup for format vdpau"). This behavior is common when there is some kind of vaapi->vdpau compatibility layer installed (on ubuntu there is the 'libvdpau-va-gl1' package), which causes ffmpeg to report vdpau as an available acceleration.
UltraGrid tries to initialize available hw. accelerations in the order ffmpeg reports them until it succeeds or falls back to sw. decoding if they all fail. So while the vdpau accel. fails, it shouldn't matter, because UltraGrid should be trying to initialize the "real" vaapi a bit later in the log. Can you confirm if this is happening?
To verify the availability of vaapi decoding, the log should contain a line like this when --verbose=7
is specified:
[1628246106.005] [lavd] Available output pixel formats: vdpau cuda vaapi_vld yuv420p
Please note that hw. decoding is usually only supported for video with 4:2:0 subsampling.
Lastly, you might need to force a specific vaapi driver. You can do this by launching UltraGrid like this:
LIBVA_DRIVER_NAME=iHD ./uv ....
Bringing this back up to the top:
Ubuntu 20. Same setup as previous tests.
I can confirm that there is no need for hw_acel becuase there is no hevc_vaapi. So my bad, its listed there very clear in codec help.
BUT
I am trying to get the encode portion sorted out. At one point in time I had this working, or at least thought it was working:
My command:
uv -t decklink:0:Hi59 -c libavcodec:encoder=hevc_vaapi:bitrate=10000k -s embedded --audio-codec=AAC:sample_rate=48000:bitrate=256k --audio-capture-format channels=2 127.0.0.1 -l unlimited -m 1316 -P 10000
The error I get:
[lavc] Warning: Unknown encoder hevc_vaapi. Using default configuration values. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder hevc_vaapi. libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so has no function __vaDriverInit_0_32 libva info: va_openDriver() returns -1 [AVHWDeviceContext @ 0x7eff54007f00] Failed to initialise VAAPI connection: -1 (unknown libva error). libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so has no function __vaDriverInit_0_32 libva info: va_openDriver() returns -1 [AVHWDeviceContext @ 0x7eff54008500] Failed to initialise VAAPI connection: -1 (unknown libva error). [hw accel] Unable to create hwdevice!! [lavc] Codec supported pixel formats: vaapi [lavc] Usable pixel formats: vaapi uyvy422 yuv422p yuvj422p uyvy422 yuv444p yuvj444p yuv420p yuvj420p nv12 yuv422p10le yuv444p10le yuv420p10le p010le yuv444p12le yuv422p16le yuv444p16le rgb24 gbrp bgr0 gbrp12le rgb48le [lavc] No direct decoder format for: UYVY. Trying to convert with swscale instead. [lavc] Unable to find suitable pixel format for: UYVY.
Any direction here would be helpful. Vainfo says all is good and I have everything I need. But its still not working.
I think that this line is important:
libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so has no function __vaDriverInit_0_32
This, according to my experiences indicates mismatch between used libva library and VAAPI driver. Do you build UG by yourself or use a AppImage? If the later one, I'd assume that driver version used is from system and not from the AppImage (which can be a problem, because it bundles own libva dependency).
In the other case, I'd suggest to check which library does UltraGrid resolve (using ldd
) and compare it eg. by the output of "ldd $(command -v vainfo) | grep libva
". Although I don't assume there will be more instances in the system.
Here is the output from ldd:
ubuntu 20 > ~: ldd $(command -v vainfo) | grep libva libva-x11.so.2 => /lib/x86_64-linux-gnu/libva-x11.so.2 (0x00007f1cf27cf000) libva-drm.so.2 => /lib/x86_64-linux-gnu/libva-drm.so.2 (0x00007f1cf27ca000) libva-wayland.so.2 => /lib/x86_64-linux-gnu/libva-wayland.so.2 (0x00007f1cf27c3000) libva.so.2 => /lib/x86_64-linux-gnu/libva.so.2 (0x00007f1cf279a000)
I have 1.6+ Jul 2nd, and 1.7 appimage release version on the system.
Both report back a similar error:
1.7 Appimage release
[lavc] Using codec: H.265, encoder: hevc_vaapi [lavc] Setting bitrate to 8000000 bps. [lavc] Warning: Unknown encoder hevc_vaapi. Using default configuration values. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder hevc_vaapi. [AVHWDeviceContext @ 0x7f741c0033c0] libva: /tmp/.mount_UltraGXvzhuS/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f741c0033c0] Failed to initialise VAAPI connection: -1 (unknown libva error). [AVHWDeviceContext @ 0x7f741c002bc0] libva: /tmp/.mount_UltraGXvzhuS/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f741c002bc0] Failed to initialise VAAPI connection: -1 (unknown libva error). [hw accel] Unable to create hwdevice!!
1.6+ Jul 2nd
[lavc] Using codec: H.265, encoder: hevc_vaapi [lavc] Setting bitrate to 8000000 bps. [lavc] Warning: Unknown encoder hevc_vaapi. Using default configuration values. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder hevc_vaapi. libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so has no function __vaDriverInit_0_32 libva info: va_openDriver() returns -1 [AVHWDeviceContext @ 0x7fc5c8003600] Failed to initialise VAAPI connection: -1 (unknown libva error). libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so has no function __vaDriverInit_0_32 libva info: va_openDriver() returns -1 [AVHWDeviceContext @ 0x7fc5c8002f40] Failed to initialise VAAPI connection: -1 (unknown libva error). [hw accel] Unable to create hwdevice!! [lavc] Codec supported pixel formats: vaapi [lavc] Usable pixel formats: vaapi uyvy422 yuv420p yuvj420p nv12 yuv420p10le p010le [lavc] No direct decoder format for: UYVY. Trying to convert with swscale instead. [lavc] Unable to find suitable pixel format for: UYVY. [lavc] Requested subsampling not supported. Try different subsampling, eg. "subsampling={420,422,444}".
It makes sense that the vaapi isnt correct, but I am not smart enough to find out how to resolve this.
I can roll back to a different ubuntu version if you think that might help.
So if I read the logs correctly. 1.6 wants VA-API "vaapi" 0.32 and I have 0.39 on the system. Is that the correct assumption?
For 1.7 I don't understand that one just yet.
Vainfo reports back that its version 1.7 libva 2.6.0
error: XDG_RUNTIME_DIR not set in the environment. error: can't connect to X server! libva info: VA-API version 1.7.0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_7 libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.7 (libva 2.6.0) vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 () vainfo: Supported profile and entrypoints VAProfileNone : VAEntrypointVideoProc VAProfileNone : VAEntrypointStats VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointFEI VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264High : VAEntrypointFEI VAProfileH264High : VAEntrypointEncSliceLP VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointFEI VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointFEI VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSlice VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD
I see, no, actually the errors are different - with the former version there was really a problem that libva inside the AppImage was incompatible with the version in system. The later version brings its own drivers and the error means that something different is wrong. Would it be possible to try h264_vaapi
for now and check if it fails as well?
Both fail with the same errors as listed in the previous post.
Am I reading this correctly?
libva info: VA-API version 1.7.0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_7 libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.7 (libva 2.6.0) vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 () vainfo: Supported profile and entrypoints
Currently libav version is 2.6.0, and API is 1.7.
When I run UG from February continuous, the error shows:
libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so has no function __vaDriverInit_0_32
Looks like it's asking for API 0.32.0, but libva says that 0.39.0, maybe thats the error?
And when I use the current UG release I get error:
[lavc] Warning: Unknown encoder hevc_vaapi. Using default configuration values. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder hevc_vaapi. [AVHWDeviceContext @ 0x7f7d0000a400] libva: /tmp/.mount_UltraG68h3ai/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f7d0000a400] Failed to initialise VAAPI connection: -1 (unknown libva error). [AVHWDeviceContext @ 0x7f7d000029c0] libva: /tmp/.mount_UltraG68h3ai/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f7d000029c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
So now I am totally lost. Any suggestions?
Both fail with the same errors as listed in the previous post.
could you please paste the output?
Also, would it be possible try to run:
ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc -vf 'format=nv12,hwupload' -c:v h264_vaapi out_h264.mp4
and
ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc -vf 'format=nv12,hwupload' -c:v hevc_vaapi out_hevc.mp4
and check the output?
Thanks
hevc error:
[lavc] Using codec: H.265, encoder: hevc_vaapi [lavc] Setting bitrate to 10000000 bps. [lavc] Warning: Unknown encoder hevc_vaapi. Using default configuration values. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder hevc_vaapi. [AVHWDeviceContext @ 0x7f3e64003600] libva: /tmp/.mount_UltraGOne8Hi/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f3e64003600] Failed to initialise VAAPI connection: -1 (unknown libva error). [AVHWDeviceContext @ 0x7f3e64002b40] libva: /tmp/.mount_UltraGOne8Hi/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f3e64002b40] Failed to initialise VAAPI connection: -1 (unknown libva error). [hw accel] Unable to create hwdevice!! [lavc] Codec supported pixel formats: vaapi [lavc] Usable pixel formats: vaapi uyvy422 yuv422p yuvj422p uyvy422 yuv444p yuvj444p yuv420p yuvj420p nv12 yuv422p10le yuv444p10le yuv420p10le p010le yuv444p12le yuv422p16le yuv444p16le rgb24 gbrp bgr0 gbrp12le rgb48le [lavc] No direct decoder format for: UYVY. Trying to convert with swscale instead. [lavc] Unable to find suitable pixel format for: UYVY.
h264 error:
[lavc] Using codec: H.264, encoder: h264_vaapi [lavc] Setting bitrate to 10000000 bps. [lavc] Warning: Unknown encoder h264_vaapi. Using default configuration values. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder h264_vaapi. [AVHWDeviceContext @ 0x7fdb2c005500] libva: /tmp/.mount_UltraGagen7J/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7fdb2c005500] Failed to initialise VAAPI connection: -1 (unknown libva error). [AVHWDeviceContext @ 0x7fdb2c005140] libva: /tmp/.mount_UltraGagen7J/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7fdb2c005140] Failed to initialise VAAPI connection: -1 (unknown libva error). [hw accel] Unable to create hwdevice!! [lavc] Codec supported pixel formats: vaapi [lavc] Usable pixel formats: vaapi uyvy422 yuv422p yuvj422p uyvy422 yuv444p yuvj444p yuv420p yuvj420p nv12 yuv422p10le yuv444p10le yuv420p10le p010le yuv444p12le yuv422p16le yuv444p16le rgb24 gbrp bgr0 gbrp12le rgb48le [lavc] No direct decoder format for: UYVY. Trying to convert with swscale instead. [lavc] Unable to find suitable pixel format for: UYVY.
I will check the ffmpeg output later today and update this.
Thanks, for the output. It doesn't write anything interesting for H.264 neither, indeed.
Just FYI, I've made couple of fixes in continuous relase today, I'd recommend to update (you can use: ./UltraGrid-continuous-x86_64.AppImage -u
), however I wouldn't expect a change here.
Anyways, I think that I was able to reproduce the problem, anyway, with a computer with Intel Xͤ graphics. That is obviously to new to be supported with the old drivers. Which CPU do you have?
I'd further suggest that what Martin proposed:
LIBVA_DRIVER_NAME=iHD ./UltraGrid-continuous-x86_64.AppImage [args]
It wouldn't work for that Intel Xͤ , but for an older CPU/GPU it might. Otherwise there are currently only 2 solutions - either you compile from sources (which will work if the above FFmpeg command works), or I can try to modify the AppImage to use system library if possible.
So first of all thanks for all the support: Here are the results.
ffmpeg reports back:
V..... h264_vaapi H.264/AVC (VAAPI) (codec h264) V..... hevc_vaapi H.265/HEVC (VAAPI) (codec hevc) V..... mjpeg_vaapi MJPEG (VAAPI) (codec mjpeg) V..... mpeg2_vaapi MPEG-2 (VAAPI) (codec mpeg2video) V..... vp8_vaapi VP8 (VAAPI) (codec vp8) V..... vp9_vaapi VP9 (VAAPI) (codec vp9)
Using the: LIBVA_DRIVER_NAME=iHD ./UltraGrid-continuous-x86_64.AppImage [args]
Results this error:
[DeckLink] Audio frame too small! [lavc] Using codec: H.265, encoder: hevc_vaapi [lavc] Setting bitrate to 10000000 bps. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder hevc_vaapi. [AVHWDeviceContext @ 0x7f8060003640] Failed to initialise VAAPI connection: -1 (unknown libva error). [AVHWDeviceContext @ 0x7f80600026c0] Failed to initialise VAAPI connection: -1 (unknown libva error). [hw accel] Unable to create hwdevice!! [lavc] Codec supported pixel formats: vaapi [lavc] Usable pixel formats: vaapi uyvy422 yuv422p yuvj422p uyvy422 yuv444p yuvj444p yuv420p yuvj420p nv12 yuv422p10le yuv444p10le yuv420p10le p010le yuv444p12le yuv422p16le yuv444p16le rgb24 gbrp bgr0 gbrp12le rgb48le [lavc] No direct decoder format for: UYVY. Trying to convert with swscale instead. [lavc] Unable to find suitable pixel format for: UYVY. [lavc] Using codec: H.265, encoder: hevc_vaapi [lavc] Setting bitrate to 10000000 bps. [lavc] Warning: Codec doesn't support slice-based multithreading. [lavc] Warning: Unable to find suitable preset for encoder hevc_vaapi. [AVHWDeviceContext @ 0x7f806079cf00] Failed to initialise VAAPI connection: -1 (unknown libva error). [AVHWDeviceContext @ 0x7f806079c4c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
CPU Details: model name : Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz
Both ffmpeg commands work perfectly with those examples you shared.
Ok, thanks for the info, that make sense. So I'll try to add dynamic system preload of libva and we will see if it works. I cannot promise now that it would but there is a decent chance. Otherwise there will currently remain only compiling from sources and linking with the system library.
Ok, after few attempts I think I've implemented the workaround, you can try the current continuous build.
It works now! YAY! Amazing. Thank you.
Yes that finally did it, whatever you fixed, I can now enable the VAAPI HEVC encode to work. Now all I need to do is find a way to make the decode work with some sort of hardware acceleration.
So adding this this a bit more info. I'm chasing the vaapi or qsv encode decode solution to offload as much power to the hardware acceleration. I was able to get qsv installed correctly as well as vaapi, but UG only shows the vaapi encoder, but no qsv encoder or decoder. I thought that the vaapi was same as qsv, but I think I am wrong. Any suggestions on this?
ffmpeg shows qsv is installed: /usr/local/ffmpeg/bin/ffmpeg -decoders | grep qsv ffmpeg version N-104708-g8a18db3ec6 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04) configuration: --prefix=/usr/local/ffmpeg --extra-cflags=-I/opt/intel/mediasdk/include --extra-ldflags=-L/opt/intel/mediasdk/lib --extra-ldflags=-L/opt/intel/mediasdk/plugins --enable-libmfx --enable-vaapi --enable-opencl --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-gpl --cpu=native --enable-libfdk-aac --enable-libx264 --enable-libx265 --extra-libs=-lpthread --enable-nonfree libavutil 57. 9.101 / 57. 9.101 libavcodec 59. 14.100 / 59. 14.100 libavformat 59. 9.102 / 59. 9.102 libavdevice 59. 0.101 / 59. 0.101 libavfilter 8. 17.100 / 8. 17.100 libswscale 6. 1.101 / 6. 1.101 libswresample 4. 0.100 / 4. 0.100 libpostproc 56. 0.100 / 56. 0.100 V....D av1_qsv AV1 video (Intel Quick Sync Video acceleration) (codec av1) V....D h264_qsv H264 video (Intel Quick Sync Video acceleration) (codec h264) V....D hevc_qsv HEVC video (Intel Quick Sync Video acceleration) (codec hevc) V....D mjpeg_qsv MJPEG video (Intel Quick Sync Video acceleration) (codec mjpeg) V....D mpeg2_qsv MPEG2VIDEO video (Intel Quick Sync Video acceleration) (codec mpeg2video) V....D vc1_qsv VC1 video (Intel Quick Sync Video acceleration) (codec vc1) V....D vp8_qsv VP8 video (Intel Quick Sync Video acceleration) (codec vp8) V....D vp9_qsv VP9 video (Intel Quick Sync Video acceleration) (codec vp9)
But UG dosnt show it and throws an error when I try to force using it: ` ~/UltraGrid-continuous-x86_64.AppImage -d decklink:device=0 --param force-lavd-decoder=h264_qsv -r embedded 127.0.0.1 -P 10000 UltraGrid 1.7+ (tags/continuous rev 9b25a389 built Nov 30 2021 16:24:02)
Display device : decklink Capture device : none Audio capture : none Audio playback : embedded MTU : 1500 B Video compression: none Audio codec : PCM Network protocol : UltraGrid RTP Audio FEC : none Video FEC : none
Created new RTP session with SSRC 0x7e13ab03. [Decklink display] Using low-latency mode. In case of problems, you can try '-d decklink:no-low-latency'. [Decklink display] Using device DeckLink SDI Micro [Decklink display] Audio output set to: default. Display initialized-decklink Video capture initialized-none Created new RTP session with SSRC 0x10025444. Audio receiving started. New incoming audio format detected: 48000 Hz, 16 channels, 32 bits per sample, codec MP3 [video dec.] New incoming video format detected: 1920x1080 @59.94i, codec H.264 [lavcd aud.] Using audio decoder: mp3float Last message repeated 15 times [lavd] Decoder not found: h264_qsv [lavd] Using decoder: h264 Audio reconfiguration succeeded (16 channels, 4 Bps, 48000 Hz, codec: PCM). [Decklink display] audio buffer underflow!`
I thought that the vaapi was same as qsv, but I think I am wrong.
Yes and no - it is true that it is run by the same circuits but those are completely different APIs.
QuickSync would work with UltraGrid, but you'd need to compile UltraGrid by yourself. Last time I tried it in July '21, it was almost impossible to add its support to the AppImage because of the structure of the library. I could upload the attempts if you wish to look at it but trust me, I've spent some 2 weeks trying it - I can write more details if you wish. Maybe we could try to check the library later if this changes but I don't think there is much improvement in few last months.
Ok, I will give that a try. I saw you had another git for compilation, is that active and possible to work off of to start? I have my own script to install QSV and it works in ffmpeg. Maybe that will be easier to do a build for UG.
I saw you had another git for compilation, is that active and possible to work off of to start?
Do you mean my personal repo? This is my development git - of course you can use it although it is not necessary - it is synced with master repo quite often (at least the devel branch) and there is additionally only work-in-progress.
I have my own script to install QSV and it works in ffmpeg. Maybe that will be easier to do a build for UG.
Ok, UltraGrid itself does use QSV through FFmpeg/libavcodec anyway. So if it is works in FFmpeg, it will work with UltraGrid as well. BTW, at least for current Ubuntu versions (& Debian) have already distro FFmpeg linked with libmfx (Intel Media SDK lib).
Yeah.... it works in ffmpeg, but still dosnt work in UG.
Yeah.... it works in ffmpeg, but still dosnt work in UG.
Some more info about that? The encoder isn't visible or something else? What does -c libavcodec:help
show? Also do you use distro ffmpeg or an own build? For the later case, it would be needed to ensure if UltraGrid links to the correct instance - what does ldd <path_to>/uv
show?
I haven't had the time to setup the proper test with ffmpeg, but I have used the default ffmpeg via apt in Ubuntu 20 and it doesn't show the qsv enabled as encode or decode, just the VAAPI version.
I had compiled manually and now I can see encode and decode QSV, but Ulgragrid Appimage still only returns that VAAP is available.
I am trying to compile UG by hand, but as you know, not an easy task and requires a little bit more than one hour an evening to address.
Any other insight would be great, but for now I am going to be 1000% sure that ffmpeg and QSV work correctly in my latest compile that has them listed.
Edit:
ldd shows an error, I am sure I am doing this wrong:
root@:~# ldd ~/UltraGrid-continuous-x86_64.AppImage not a dynamic executable root@:~#
1) I wonder if it would be useful to add intel-media-va-driver-non-free to the AppImage.
2) Here is what I do to compile UG on Ubuntu, although I tend to use the AppImage now.
sudo apt-get -y install build-essential autoconf automake pkgconf libtool git cmake freeglut3 freeglut3-dev uuid-dev
sudo apt-get -y install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libva-dev libvdpau-dev libglib2.0-dev libcurl4-nss-dev libssl-dev libv4l-dev libsdl2-dev libglew-dev libswscale-dev libx265-dev libx265-doc libasound-dev libmfx1 vainfo
sudo apt-get -y install alsa alsa-base x11-xserver-utils apulse pulseaudio intel-media-va-driver-non-free i965-va-driver-shaders
git clone https://github.com/CESNET/UltraGrid.git
cd ~/UltraGrid && git submodule init && git submodule update
cd ~/UltraGrid/cineform-sdk/ && cmake . && make CFHDCodecStatic
cd ~/UltraGrid
./autogen.sh --enable-plugins
make -jnproc
Thanks for that, that works well. But I still don't see qsv in the libavhelp
./uv -c libavcodec:help UltraGrid 1.7+ (tags/continuous rev c59aabc8 built Dec 9 2021 18:29:22)
Display device : none Capture device : none Audio capture : none Audio playback : none MTU : 9000 B Video compression: libavcodec:help Audio codec : PCM Network protocol : UltraGrid RTP Audio FEC : none Video FEC : none
Display initialized-none
Video capture initialized-none
Libavcodec encoder usage:
-c libavcodec[:codec=
where
@alatteri Thanks for the info. I think we can consider that later but currently UltraGrid uses U18.04 for the build which doesn't that package and I'd perhaps avoid building the libva stuff extra distribution packages. On the other hand, as currently it is implemented dynamic loading of system libva if it is installed, the user can quite easily install the packages to the system and UG would utilize that.
@TheSashmo
Thanks for that, that works well. But I still don't see qsv in the libavhelp
The libavcodec/FFmpeg library used is still the one from the AppImage, without QSV. As indicated above on 1st Dec (notes), it is uncertain whether system QSV (libmfx) library can be used. If you want to try QuickSync, I'd suggest compiling UG from source. In Ubuntu 21.10 and Debian 11 is FFmpeg linked with libmfx if you won't bother with compiling FFmpeg (in Ubuntu 20.04 it isn't, but libmfx is present).
I cannot now promise adding QSV, at least in the near future, since AppImages are based on U18.04 and I didn't succeeded trying this. I think that the amount of work is huge (with uncertain results) while it is uncertain to me how better this library is than using just libva (maybe more options/presets?)
So just bringing this back up to the top again, and trying to get vaapi/qsv hardware accelerated encoding and its just not working.
OS: Ubuntu 20 Ultragrid: 1.7.6
Command line example: ./UltraGrid-1.7.6-x86_64.AppImage -t testcard:2160:3840:60:YUYV -c libavcodec:encoder=h264_vaapi:bitrate=25000k 127.0.0.1 -l unlimited -m 1316 -P 10000
Error: [AVHWDeviceContext @ 0x7f2b5400f6c0] libva: /tmp/.mount_UltraGkbHXrl/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f2b5400f6c0] Failed to initialise VAAPI connection: -1 (unknown libva error). [AVHWDeviceContext @ 0x7f2b5400aa00] libva: /tmp/.mount_UltraGkbHXrl/usr/lib/va/i965_drv_video.so init failed [AVHWDeviceContext @ 0x7f2b5400aa00] Failed to initialise VAAPI connection: -1 (unknown libva error).
Vainfo: libva info: VA-API version 1.7.0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_7 libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.7 (libva 2.6.0) vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 () vainfo: Supported profile and entrypoints VAProfileNone : VAEntrypointVideoProc VAProfileNone : VAEntrypointStats VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointFEI VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264High : VAEntrypointFEI VAProfileH264High : VAEntrypointEncSliceLP VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointFEI VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointFEI VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSlice VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD
Any suggestions here on what I am missing?
Try installing intel-media-va-driver-non-free i965-va-driver-shaders
Already installed:
i965-va-driver-shaders is already the newest version (2.4.0-1build1). intel-media-va-driver-non-free is already the newest version (20.1.1+ds1-1build1).
libmfx-dev
. maybe? Are you running headless of with a GUI?
Yes I am headless. I installed that, but still no luck. Same error.
I wonder if it need a "GUI" or Xorg or something to actually be running to engage the driver?
Just tried it with x11 forwarding and the result it the same. I validated that x11 is working by starting firefox first, so I know that it should be kicking in if that was it.
I just tried encoding vappi on Ubuntu 22.04 with the a latest continuous and it failed too.
Command args: -t testcard:1920:1080:24:RGB -c libavcodec:encoder=hevc_vaapi --control-port 36661 --param errors-fatal
UltraGrid 1.7+ (tags/continuous rev 574b5fe1 built Oct 7 2022 14:29:37)
Display device : none
Capture device : testcard
Audio capture : none
Audio playback : none
MTU : 1500 B
Video compression: libavcodec:encoder=hevc_vaapi
Audio codec : PCM
Network protocol : UltraGrid RTP
Audio FEC : none
Video FEC : none
Display initialized-none
[testcard] capture set to 1920x1080 @24.00p, codec RGB, bpp 3, pattern: bars, audio off
Video capture initialized-testcard
[key control] Stdin is not a TTY - disabling keyboard control.
Created new RTP session with SSRC 0x43a8445a.
[lavc] Using codec: H.265, encoder: hevc_vaapi
[lavc] Setting CQP to 21
[lavc] Selected pixfmt: nv12
[lavc hevc_vaapi @ 0x7f0acc001d40] Failed to end picture encode issue: 24 (internal encoding error).
[lavc hevc_vaapi @ 0x7f0acc001d40] Encode failed: -5.
[lavc] Error encoding frame: Input/output error
[lavc] Assertion pic->display_order == pic->encode_order failed at libavcodec/vaapi_encode_h265.c:765
Backtrace:
/tmp/.mount_UltraGICOeCf/usr/bin/uv(+0x1088fb)[0x5614705088fb]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f0b173f0520]
/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7f0b17444a7c]
/lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7f0b173f0476]
/lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7f0b173d67f3]
/tmp/.mount_UltraGICOeCf/usr/lib/libavcodec.so.59(+0x745c42)[0x7f0b09145c42]
/tmp/.mount_UltraGICOeCf/usr/lib/libavcodec.so.59(+0x73e504)[0x7f0b0913e504]
/tmp/.mount_UltraGICOeCf/usr/lib/libavcodec.so.59(+0x7407a9)[0x7f0b091407a9]
/tmp/.mount_UltraGICOeCf/usr/lib/libavcodec.so.59(+0x2ba427)[0x7f0b08cba427]
/tmp/.mount_UltraGICOeCf/usr/lib/libavcodec.so.59(avcodec_send_frame+0xc7)[0x7f0b08cba7b7]
/tmp/.mount_UltraGICOeCf/usr/bin/../lib/ultragrid/ultragrid_vcompress_libavcodec.so(+0x28e88)[0x7f0af9028e88]
/tmp/.mount_UltraGICOeCf/usr/bin/uv(+0xdf3a4)[0x5614704df3a4]
/tmp/.mount_UltraGICOeCf/usr/bin/uv(_ZN9wp_worker3runEv+0x9d)[0x5614704ceecd]
/tmp/.mount_UltraGICOeCf/usr/bin/uv(_ZN9wp_worker10enter_loopEPv+0x18)[0x5614704cf008]
/lib/x86_64-linux-gnu/libc.so.6(+0x94b43)[0x7f0b17442b43]
/lib/x86_64-linux-gnu/libc.so.6(+0x126a00)[0x7f0b174d4a00]
UltraGrid has crashed (Aborted).
Please send a bug report to address ultragrid-dev@cesnet.cz.
You may find some tips how to report bugs in file doc/REPORTING_BUGS.md distributed with UltraGrid
(or available online at https://github.com/CESNET/UltraGrid/blob/master/doc/REPORTING-BUGS.md).
Process exited with code: 6
@TheSashmo you must be still using the continuous build. There has been no major release since the end of the last year.
@alatteri are there any conditions/steps to reproduce the issue? Using the continuous version in Ubuntu 22.04 with intel-media-va-driver-non-free installed (required) worked flawlessly for me both under X11 and Wayland with HD Graphics 530.
I think that test was done headless Ubuntu Server ....I try it again on a machine with GUI installed.
@alatteri are there any conditions/steps to reproduce the issue? Using the continuous version in Ubuntu 22.04 with intel-media-va-driver-non-free installed (required) worked flawlessly for me both under X11 and Wayland with HD Graphics 530.
Not sure if this is something that can be addressed but I am using these examples:
uv -t decklink:0:Hi59 --param lavc-use-codec=vaapi_vld -c libavcodec:encoder=hevc_vaapi:bitrate=10000k -s embedded --audio-codec=AAC:sample_rate=48000:bitrate=256k --audio-capture-format channels=2 127.0.0.1 -l unlimited -m 1316 -P 10000
and
uv -d decklink:device=0 --param force-lavd-decoder=hevc_vaapi -r embedded 127.0.0.1 -P 10000
I don't see any dropped frames in the decode, its clean. No errors. The encode shows 150 frames in 5 or so seconds.... All seems to be good, but if I check the quality of the output on the decoded video there is at times in some motion dropped frames. I can see it skip some frames.
Any suggestions, or just give up on quicksync?