QSV intel hardware acceleration - video storage (endocer) AV1

[Arol450]

Environment: ZM: V1.36.33 tag with modifications to source FFMPEG: N-109920-gac6eec1fc2 (GIT) with --enable-libvpl compiled Kernel: 6.2.0-1-drm-tip-git-g798a0dbb51e8 Graphics card: Intel ARC A770

I have FFMPEG working very well with QSV hardware support for decoding (HEVC, H264) and encoding to AV1 via av1_qsv.

ffmpeg -hwaccel qsv -c:v hevc_qsv -i someHEVCinput.mp4 -c:v av1_qsv -profile:v main -preset veryslow -qscale 75 output.mp4

works perfectly and amazingly quickly. It produces very high quality videos with stunningly small file sizes.

I have a 15+ IP cameras delivering either H264 or H265 videos and I would like to store the videos after zoneminder is done with them in AV1 (for the above reasons).

So I made a couple of changes to the source code (inspired by: https://forums.zoneminder.com/viewtopic.php?t=31543 and https://github.com/exuvo/zoneminder/commit/9fd0a94c9f85e2dbec6ff8ac3f219332abf95ebd):

To zm_videostore.cpp i added the lines with a plus (+) { AV_CODEC_ID_MJPEG, "mjpeg", "mjpeg", AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ422P, AV_HWDEVICE_TYPE_NONE }, +{ AV_CODEC_ID_VP9, "vp9", "libvpx-vp9", AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV420P, AV_HWDEVICE_TYPE_NONE }, +{ AV_CODEC_ID_AV1, "av1", "libsvtav1", AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV420P, AV_HWDEVICE_TYPE_NONE }, +{ AV_CODEC_ID_AV1, "av1", "libaom-av1", AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV420P, AV_HWDEVICE_TYPE_NONE }, +{ AV_CODEC_ID_AV1, "av1", "av1_qsv", AV_PIX_FMT_NV12, AV_PIX_FMT_NV12, AV_HWDEVICE_TYPE_NONE }, ELSE{ AV_CODEC_ID_H265, "h265", "libx265", AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV420P }

To monitor.js: +} else if ( this.value == 167 ) { +option.disabled = !(option.value.includes('vp9')); +if ( option.disabled && option.selected ) { +encoder_dropdown[0].options[0].selected = 1; +option.selected = false; +} } else if ( this.value == 173 ) { option.disabled = !(option.value.includes('hevc') || option.value.includes('265') ); if ( option.disabled && option.selected ) { encoder_dropdown[0].options[0].selected = 1; option.selected = false; } +} else if ( this.value == 226 ) { +option.disabled = !(option.value.includes('av1')); +if ( option.disabled && option.selected ) { +encoder_dropdown[0].options[0].selected = 1; +option.selected = false; +} } else { option.disabled = false;

To monitor.php: $videowriter_codecs = array( '0' => translate('Auto'), '27' => 'h264', '173' => 'h265/hevc', +'167' => 'vp9', +'226' => 'av1', );

AND

$videowriter_encoders = array( 'auto' => translate('Auto'), 'libx264' => 'libx264', 'h264' => 'h264', 'h264_nvenc' => 'h264_nvenc', 'h264_omx' => 'h264_omx', 'h264_vaapi' => 'h264_vaapi', 'libx265' => 'libx265', 'hevc_nvenc' => 'hevc_nvenc', 'hevc_vaapi' => 'hevc_vaapi', +'libvpx-vp9' => 'libvpx-vp9', +'libsvtav1' => 'libsvtav1', +'libaom-av1' => 'libaom-av1', +'av1_qsv' => 'av1_qsv', );

So far, so good. Compiled nicely. Started nicely. System load is huge as QSV is not supported as a decoder option in zoneminder (that's a different topic) but I get videos of all my cameras.

YET: I do not get video stored in AV1 with hardware acceleration at all. Logs: zmc_m10 | WAR | Failed to open videostore, turning on jpegs | zm_event.cpp zmc_m10 |   ERR | Can't open video codec! | zm_videostore.cpp | 347 zmc_m10 |   WAR | Can't open video codec (av1_qsv) Function not implemented

What am I doing wrong? Would somebody please be so nice to point me in the right direction or help me? Thank you!

[welcome[bot]]

Thanks for opening your first issue here! Just a reminder, this forum is for Bug Reports only. Be sure to follow the issue template!

[Arol450]

Just compiled from master (1.36.34), made similar changes to the source code. This time I get a slightly different error message: zmc_m11 | WAR | Failed to open videostore, turning on jpegs | zm_event.cpp zmc_m11 | ERR | Can't open video codec! | zm_videostore.cpp zmc_m11 | WAR | Can't open video codec (av1_qsv) Invalid argument

[connortechnology]

Turn on debug logging to get more information.

[connortechnology]

None of the devs have an ARC so at this time not much help available.

[connortechnology]

I have added your additional lines, with what I think are some fixes, but without hardware support.. going to be hard for me to help further. it may be possible for me to get qsv_h264/hevc going and maybe that will shed some light.

[Arol450]

Here is some debug output: zmc_m11[37092].DB1-zm_videostore.cpp/116 [Opening video storage stream /var/lib/zoneminder/events/11/2023-02-25/69/incomplete.mp4 format: mp4] zmc_m11[37092].DB2-zm_videostore.cpp/268 [Codec wanted 226 av1] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name hevc_vaapi != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name hevc_nvenc != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name libx265 != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name h264_vaapi != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name h264_nvenc != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name h264_omx != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name h264 != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name libx264 != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name mjpeg != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name libvpx-vp9 != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name libsvtav1 != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/276 [Not the right codec name libaom-av1 != av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/295 [Found video codec for av1_qsv] zmc_m11[37092].DB1-zm_videostore.cpp/305 [Setting pix fmt to 23 nv12] zmc_m11[37092].WAR-zm_videostore.cpp/373 [Can't open video codec (av1_qsv) Function not implemented] zmc_m11[37092].ERR-zm_videostore.cpp/413 [Can't open video codec!] zmc_m11[37092].WAR-zm_event.cpp/658 [Failed to open videostore, turning on jpegs]

[Arol450]

I will go ahead and pull the latest from git and give the hevc_qsv a try, as well. Thanks for your help. I would gladly volunteer to set up a test-machine for this platform if that is something that would be helpful for the project.

[Arol450]

okay, just pulled the latest and greatest with your changes. I get these error messages: zmc_m11[69990].DB2-zm_ffmpeg.cpp/68 [Using the variable bitrate (VBR) ratecontrol method] zmc_m11[69990].WAR-zm_ffmpeg.cpp/68 [Current frame rate is unsupported] zmc_m11[69990].WAR-zm_ffmpeg.cpp/68 [some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.] ... zmc_m11[69990].WAR-zm_videostore.cpp/374 [Can't open video codec (av1_qsv) Function not implemented] zmc_m11[69990].WAR-zm_videostore.cpp/389 [Encoder Option look_ahead not recognized by ffmpeg codec] zmc_m11[69990].ERR-zm_videostore.cpp/414 [Can't open video codec!] zmc_m11[69990].WAR-zm_event.cpp/658 [Failed to open videostore, turning on jpegs]

The "Function not implemented" warning threw me off a bit at first, while it just seems to be an error that is QSV related. I am not too sure what to do about that atm. Any input would be welcomed. (I have tried with several codec options, including, and without, look_ahead).

Trying HEVC next (which is not supported by Firefox, hence I wanted to transition to AV1 in the first place... ).

[Arol450]

So, this time with hevc_qsv, (profile=1 scenario=6 preset=1).

zmc_m11[71526].DB2-zm_ffmpeg.cpp/68 [Use Intel(R) oneVPL to create MFX session with the specified MFX loader] zmc_m11[71516].DB1-zm_packet.cpp/124 [got frame: format 12 yuvj420p 1920x1080 linesize:1920x960 pts: 1208430 keyframe: 0] zmc_m11[71516].DB3-zm_packet.cpp/203 [Same pix format yuvj420p so not hwtransferring. sw_pix_fmt is yuvj420p] zmc_m11[71516].DB1-zm_image.cpp/196 [line size: 7680 =? 7680 width 1920 Size 8294400 ?= 8294400] zmc_m11[71516].DB1-zm_image.cpp/265 [PopulateFrame: width 1920 height 1080 linesize 7680 colours 4 imagesize 8294400 rgba] zmc_m11[71516].DB1-zm_image.cpp/294 [Image.Populate(frame): format 26 rgba 1920x1080 linesize:7680x0 pts: -9223372036854775808 keyframe: 1] zmc_m11[71516].DB1-zm_image.cpp/326 [source frame before convert: format 12 yuvj420p 1920x1080 linesize:1920x960 pts: 1208430 keyframe: 0] zmc_m11[71516].DB1-zm_image.cpp/329 [Assign src linesize: 1920, dest linesize: 7680] ... zmc_m11[71526].DB2-zm_ffmpeg.cpp/68 [Using the variable bitrate (VBR) ratecontrol method] zmc_m11[71526].WAR-zm_ffmpeg.cpp/68 [Low power mode is unsupported] zmc_m11[71526].WAR-zm_ffmpeg.cpp/68 [some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.] ... zmc_m11[71526].WAR-zm_videostore.cpp/374 [Can't open video codec (hevc_qsv) Function not implemented] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name hevc_nvenc != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name libx265 != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name h264_vaapi != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name h264_qsv != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name h264_nvenc != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name h264_omx != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name h264 != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name libx264 != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name mjpeg != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name libvpx-vp9 != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name libsvtav1 != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name libaom-av1 != hevc_qsv] zmc_m11[71526].DB1-zm_videostore.cpp/277 [Not the right codec name av1_qsv != hevc_qsv] zmc_m11[71526].ERR-zm_videostore.cpp/414 [Can't open video codec!] zmc_m11[71526].WAR-zm_event.cpp/658 [Failed to open videostore, turning on jpegs]