Open walterav1984 opened 4 years ago
Out of curiosity, what does the log throw if you create a channel with vaapi in the casparcg.conf file? And could you provide the arguments used in the config file when you do that please?
Both "casparcg.config" and "AMCP" error quiet similar, updated my original topic with "casparcg.config" ffmpeg consumer arguments @LRTNZ .
@walterav1984 reading the FFmpeg wiki it says "hardware generally wants the nv12 layout, but most software functions use the yuv420p layout" - have you tried yuv420p?
Changing the "pre" filter format from "nv12" to "yuv420p" doesn't change the error log @LRTNZ , I guess CasparCG server feeds pure 422 or 444 to its ffmpeg consumer.
If we look closer to the log it already errors first mentioning the hardware device which cannot be found but, strange thing is its called 2 times like its passing the filter options to the vaapi_device argument?
[2020-04-30 15:13:21.035] [error] [ffmpeg] [hwupload @ 0x7f7e45ee7d40] A hardware device reference is required to upload frames to.
[2020-04-30 15:13:21.035] [error]
[2020-04-30 15:13:21.035] [error] [ffmpeg] [Parsed_hwupload_1 @ 0x7f7e7a803dc0] Query format failed for 'Parsed_hwupload_1': Invalid argument
[2020-04-30 15:13:21.035] [error]
[2020-04-30 15:13:21.035] [error] [ffmpeg] [hwupload @ 0x7f7e45ee7d40] A hardware device reference is required to upload frames to.
[2020-04-30 15:13:21.035] [error]
[2020-04-30 15:13:21.035] [error] [ffmpeg] [Parsed_hwupload_1 @ 0x7f7e7a803dc0] Query format failed for 'Parsed_hwupload_1': Invalid argument
[2020-04-30 15:13:21.035] [error]
@walterav1984 poking around a few examples of using vaapi in C++ online, and taking a quick look in the Caspar code, I am not suprised this hasn't worked out of the box.
Unlike some of the other hardware accelerated solutions, that just 'slot in' with more standard ffmpeg parameters like nvenc encoding does, vaapi requires a bit more configuration with options I do not think Caspar is setup to handle yet.
Also, it looks like according to the ffmpeg hwaccel documentation that vaapi is only is supported on Linux at this time: https://trac.ffmpeg.org/wiki/HWAccelIntro
So for this to be added in, something would need to be added to check that Caspar is not on windows when trying to use vaapi (Which wouldn't be to hard I would think), and to have a sensible fallback if that is the case.
The devs will have a better idea than me on the work required to add this in, but I believe from looking at the code (Albeit with only a little cpp experience), that it would require quite a sizable refactoring to implement this feature.
Potentially in the future, depending on the release of Intel dedicated GPUs and what they offer in the way of hardware encoding, this might become something more feasible to consider then.
@walterav1984 poking around a few examples of using vaapi in C++ online, and taking a quick look in the Caspar code, I am not suprised this hasn't worked out of the box.
Unlike some of the other hardware accelerated solutions, that just 'slot in' with more standard ffmpeg parameters like nvenc encoding does, vaapi requires a bit more configuration with options I do not think Caspar is setup to handle yet.
Was hoping that just some input sanitation would be enough, but the "pre-" "format" option that must be applied to the VAAPI device itself and perhaps "again" to the output itself is indeed confusing if CasparCG expects it a single time.
Also, it looks like according to the ffmpeg hwaccel documentation that vaapi is only is supported on Linux at this time: https://trac.ffmpeg.org/wiki/HWAccelIntro So for this to be added in, something would need to be added to check that Caspar is not on windows when trying to use vaapi (Which wouldn't be to hard I would think), and to have a sensible fallback if that is the case.
Invalid or wrong command handling is currently already erratic with CasparCG Server, but indeed VAAPI is missing for Windows and commands containing it will probably fail/error.
The devs will have a better idea than me on the work required to add this in, but I believe from looking at the code (Albeit with only a little cpp experience), that it would require quite a sizable refactoring to implement this feature.
Potentially in the future, depending on the release of Intel dedicated GPUs and what they offer in the way of hardware encoding, this might become something more feasible to consider then.
Perhaps going the more complex "libmfx" route with Intel (Broadwell Gen8) and future dedicated GPU's ahead would be a better fit for both Linux and Windows support. Installing the libmfx driver part for Ubuntu 20.04 is easy but it still needs ffmpeg to be recompiled with --enable-libmfx
which can be done in the FFmpeg docker part.
The Windows zeranoe FFmpeg builds already contain --enable-libmfx
but sadly I have no Intel Gen8 to test it, still libmfx uses VAAPI on Linux as a backend and similar syntax which investigating benefit both solutions.
Description
Building ffmpeg with
--enable-vaapi
would allow hardware accelerated encoding (decoding not interested yet) of mpeg2/h264/vc1/h265/vp8/vp9/mjpeg/etc on supported Intel hardware (Haswell graphics gen7.5 4200/4400/4600 or higher have OpenGL 4.6 support) and AMDGPU devices (experimental according to FFmpeg). Doing this encoding on the GPU would free CPU for other intensive tasks. Although this works for FFmpeg this doesn't work yet for CasparCG server.Solution suggestion
Like #1282 open the discussion to make all the pieces fall into place. Building ffmpeg with
--enable-vaapi
is not the challenge but parsing the right parameters/syntax in CasparCG Server "casparcg.config" or "AMCP" is with specifying:vaapi_device /dev/dri/renderD128
and probably-vf 'format=nv12,hwupload'
which applies to/before the vaapi device instead of having effect on the output file/stream!Scope
This request is for the basic VAAPI support, since VAAPI can be made complex by using proprietary/partly closed intelhybrid qsv / libmfx driver (Broadwell gen 8 or higher) which allows better quality and more options, but that will be part of a other pull request (with a special build flag)!
Additional information
For quick testing both Debian Buster 10 and Ubuntu 20.04 ship a CasparCG server with --enable-vaapi FFmpeg by default.
The FFmpeg wiki shows that the specifying the
vaapi_device /dev/dri/renderD128
goes before the-i input file/source
which is a pipe in CasparCG server, however the following command works with ffmpeg cli with specifyingvaapi_device
after the-i
which may work with CasparCG but it doesn't:FFmpeg default apt versions of Ubuntu 18.04/20.04
CasparCG casparcg.config consumer fails parsing
CasparCG AMCP fails parsing
CasparCG server fail parsing log (casparcg.config / AMCP similar)