[Bug]: VA_RC_ICQ not available in VP9 encoder on DG2

danielsmakandra commented 9 months ago

Which component impacted?

Encode

Is it regression? Good in old configuration?

No, this issue exist a long time

What happened?

Linux, FFmpeg self built with --enable-libvpl (version N-112750-g6d60cc7baf)
libva 2.19, intel-media-va-driver-non-free 23.3.2
Run sudo ffmpeg -v verbose -hwaccel qsv -hwaccel_output_format qsv -an -c:v h264_qsv -i ~/videos/V2023CW0079.mp4 -c:v vp9_qsv -global_quality 45 ~/output/V2023CW0079_icq.webm

Error:

[vp9_qsv @ 0x56191eae4bc0] Using the intelligent constant quality (ICQ) ratecontrol method
[vp9_qsv @ 0x56191eae4bc0] Error initializing the encoder: device failed (-17)
[vost#0:0/vp9_qsv @ 0x56191eb2d340] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.

I'm aware of https://github.com/intel/media-driver/issues/1597 and the corresponding patches but it only focused on AV1.

What's the usage scenario when you are seeing the problem?

Transcode for media delivery

What impacteded

We need to encode video to VP9 and want to use GPU encode to speed up the transcoding. But since file size/ bandwith is important we want the maximum quality for low file sizes. The ICQ Rate control mode should give us the best results but doesn't work.

We also can't use AV1 because the hardware decode support for AV1 is still not very broad.

Debug Information

vainfo -a lists VA_RC_ICQ as available but it doesn't work with ffmpeg via vpl

VAProfileVP9Profile3/VAEntrypointEncSliceLP
    VAConfigAttribRTFormat                 : VA_RT_FORMAT_YUV420
                                             VA_RT_FORMAT_YUV444
                                             VA_RT_FORMAT_YUV420_10
                                             VA_RT_FORMAT_YUV444_10
                                             VA_RT_FORMAT_RGB32
                                             VA_RT_FORMAT_RGB32_10
                                             VA_RT_FORMAT_RGB32_10BPP
                                             VA_RT_FORMAT_YUV420_10BPP
    VAConfigAttribRateControl              : VA_RC_CBR
                                             VA_RC_VBR
                                             VA_RC_CQP
                                             VA_RC_ICQ

This is on an intel Arc A380 (DG2)

Do you want to contribute a patch to fix the issue?

None

intel-mediadev commented 9 months ago

Auto Created VSMGWL-70035 for further analysis.

leyu-yao commented 9 months ago

It seems VP9 missed to map VPL ICQ enum to vaapi ICQ at here... https://github.com/oneapi-src/oneVPL-intel-gpu/blob/main/_studio/mfx_lib/encode_hw/vp9/src/mfx_vp9_encode_hw_vaapi.cpp#L61 I will prepare a patch to fix it.

dsummer commented 7 months ago

It seems VP9 missed to map VPL ICQ enum to vaapi ICQ at here... https://github.com/oneapi-src/oneVPL-intel-gpu/blob/main/_studio/mfx_lib/encode_hw/vp9/src/mfx_vp9_encode_hw_vaapi.cpp#L61 I will prepare a patch to fix it.

Any update on this? if we know what's needed to get ICQ mode working on VP9 (very important for my use case) any possibility this could be bumped to P1 as this same issue for AV1 was? VP9 has much broader compatibility vs AV1 for now, so I'd argue it's just as important.

leyu-yao commented 6 months ago

It seems VP9 missed to map VPL ICQ enum to vaapi ICQ at here... https://github.com/oneapi-src/oneVPL-intel-gpu/blob/main/_studio/mfx_lib/encode_hw/vp9/src/mfx_vp9_encode_hw_vaapi.cpp#L61 I will prepare a patch to fix it.

Any update on this? if we know what's needed to get ICQ mode working on VP9 (very important for my use case) any possibility this could be bumped to P1 as this same issue for AV1 was? VP9 has much broader compatibility vs AV1 for now, so I'd argue it's just as important.

Hi @dsummer, here's the patch to fix the above issue in VPL RT https://github.com/oneapi-src/oneVPL-intel-gpu/pull/317. But during my test, I also found some potential issues in media-driver or huc firmware. I can get the encoded bitstream but the quality factor does not actually take effect. Even if we got ICQ work for vp9, its quality will be another major concern. AVC/HEVC has well support for ICQ. If it is possible, we highly recommend to switch to avc/hevc, or newer av1.

danielsmakandra commented 6 months ago

Hi @dsummer, here's the patch to fix the above issue in VPL RT oneapi-src/oneVPL-intel-gpu#317. But during my test, I also found some potential issues in media-driver or huc firmware. I can get the encoded bitstream but the quality factor does not actually take effect. Even if we got ICQ work for vp9, its quality will be another major concern. AVC/HEVC has well support for ICQ. If it is possible, we highly recommend to switch to avc/hevc, or newer av1.

Hi @leyu-yao , thank you very much for preparing the fix. I already noticed the same issue you mentioned while trying to encode with ffmpeg and vaapi directly instead of quicksync. The set quality doesn't affect the output file at all. I reported the problem here: https://github.com/intel/libva/issues/776 (perhaps in the wrong place).

Could you lead me to the right place to report this issue so that maybe it will get fixed in the future.

Considering switching to other Codecs:

VP9 would offer higher quality at lower bitrates compared to AVC. VP9 was also the main reason for us to buy the ARC GPUs as Intel is the only vendor offering hardware accellerated transcoding. It's quite frustrating to always get the answer to switch to another codec when the reason for buying the GPUs in the first place was VP9.
HEVC has afaik licensing issues which makes it hard to use it in our use case (commercially). Hardware decode support is also not very broad
AV1 is still "too new" as only brand new devices offer hardware decoding for this codec. So while it's great, that the GPUs support it, it's not a viable codec for use (yet)

dsummer commented 6 months ago

As @danielsmakandra commented before me, VP9 support is the primary reason we've been evaluating Intel GPU as a possible transcoding platform, and what differentiates Intel GPU's from other options. AVC is more than 20 years old now, and HEVC is highly encumbered (see Broadcom v. Netflix). AV1 is clearly the future, but it will be years before a majority of the public has AV1 capable devices. VP9 is the way to go for online video that has to reach a large audience, and VP9 encode support is a standout feature that differentiates Intel GPUs from other hardware encode options. Not finishing the software when the hardware is done seems to me to be a big missed opportunity for Intel. I would strongly encourage folks @ Intel working on this to make ICQ support for VP9 P1.

It seems VP9 missed to map VPL ICQ enum to vaapi ICQ at here... https://github.com/oneapi-src/oneVPL-intel-gpu/blob/main/_studio/mfx_lib/encode_hw/vp9/src/mfx_vp9_encode_hw_vaapi.cpp#L61 I will prepare a patch to fix it.

Any update on this? if we know what's needed to get ICQ mode working on VP9 (very important for my use case) any possibility this could be bumped to P1 as this same issue for AV1 was? VP9 has much broader compatibility vs AV1 for now, so I'd argue it's just as important.

Hi @dsummer, here's the patch to fix the above issue in VPL RT oneapi-src/oneVPL-intel-gpu#317. But during my test, I also found some potential issues in media-driver or huc firmware. I can get the encoded bitstream but the quality factor does not actually take effect. Even if we got ICQ work for vp9, its quality will be another major concern. AVC/HEVC has well support for ICQ. If it is possible, we highly recommend to switch to avc/hevc, or newer av1.

intel / media-driver