[Bug]: VA_RC_ICQ not available in AV1 encoder on DG2

[nyanmisaka]

Which component impacted?

Encode

Is it regression? Good in old configuration?

No, this issue exist a long time

What happened?

1. Linux, FFmpeg
2. libva 2.16, media-driver 22.6.4 or master
3. Run ffmpeg -f lavfi -i "testsrc2=s=1920x1080,format=nv12" -c:v av1_qsv -global_quality 25 -g:v 120 -f null -
4. Error:

   [av1_qsv @ 0x55f3edf29ec0] Initialized an internal MFX session using hardware accelerated implementation
   [av1_qsv @ 0x55f3edf29ec0] Using the intelligent constant quality (ICQ) ratecontrol method
   [av1_qsv @ 0x55f3edf29ec0] Selected ratecontrol mode is unsupported
   [av1_qsv @ 0x55f3edf29ec0] Low power mode is unsupported
   [av1_qsv @ 0x55f3edf29ec0] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
   Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

What's the usage scenario when you are seeing the problem?

Transcode for media delivery

What impacted?

ICQ rate control is not available in AV1 encoder on DG2 on Linux.

Cannot use the -global_quality option in FFmpeg.

Debug Information

I tried ICQ rate control with AV1 on Windows and it works just fine with FFmpeg.

Do you want to contribute a patch to fix the issue?

None

[nyanmisaka]

Same issues:

https://github.com/oneapi-src/oneVPL/issues/82#issuecomment-1373392135

https://bbs.archlinux.org/viewtopic.php?pid=2077670#p2077670

[intel-mediadev]

Auto Created VSMGWL-60768 for further analysis.

[channeladam]

Is there any way we can see or be informed of the status/progress of VSMGWL-60768 ?

[vid-bin]

Is there any update to this?

[channeladam]

Hi, can someone please provide an update on this issue? Is there a fix in the works? Will it be addressed? Is there an expected timeline? Anything?

[nyanmisaka]

@channeladam It seems Intel haven't figure out the AV1 ICQ support on DG2 platforms, including the next gen MTL, which is being actively developed in their new code path "media_softlet".

HEVC ICQ on DG2 might just be a capability reporting bug (if DG2 and Gen12 share the same code path), while AV1 ICQ doesn't seem to be implemented yet as I can't find it anywhere.

@Jexu may be able to provide more progress on this.

[channeladam]

Thank you @nyanmisaka, I appreciate your response and you creating this issue.

I bought my Intel Arc A770 reasonably expecting/assuming HEVC & AV1 ICQ to be working since H264 ICQ does... I guess I was either foolish or misled.

It would be nice to hear from someone at Intel to understand if AV1 ICQ will happen & when, or not...
@Jexu @thomasli21801 @XinfengZhang

Thank you in advance.

[vid-bin]

@channeladam I also bought the A380 for VP9/AV1 video encoding (VP9 appears to have the same issue BTW). It would be great to get some kind of communication on this issue.

[nyanmisaka]

@channeladam I also bought the A380 for VP9/AV1 video encoding (VP9 appears to have the same issue BTW). It would be great to get some kind of communication on this issue.

The RC modes of VP9 are correct even though I don't use it.

VAProfileVP9Profile2/VAEntrypointEncSliceLP
    VAConfigAttribRTFormat                 : VA_RT_FORMAT_YUV420
                                             VA_RT_FORMAT_YUV444
                                             VA_RT_FORMAT_YUV420_10
                                             VA_RT_FORMAT_YUV444_10
                                             VA_RT_FORMAT_RGB32
                                             VA_RT_FORMAT_RGB32_10
                                             VA_RT_FORMAT_RGB32_10BPP
                                             VA_RT_FORMAT_YUV420_10BPP
    VAConfigAttribRateControl              : VA_RC_CBR
                                             VA_RC_VBR
                                             VA_RC_CQP
                                             VA_RC_ICQ
                                             VA_RC_MB

[nyanmisaka]

1630 ONLY fixes the HEVC ICQ.

AV1 ICQ needs extra handling from Intel. So don't close this issue.

[vid-bin]

The RC modes of VP9 are correct even though I don't use it.

Yet it doesn't work for me. Maybe it's user error or one of my packages it outdated, idk.

When you try running it yourself does it work?

[nyanmisaka]

When you try running it yourself does it work?

What ffmpeg command did you use?

[vid-bin]

What ffmpeg command did you use?

Just a basic ffmpeg command. Basically the same as in OP only with vp9_qsv. I also tried running -crf and -qp.

I then tried to use vp9_qsv with peertube in VBR. av1_qsv works fine in VBR. The 30 minute VP9 output works for 2-3 minutes and then seeking stops working and the video corrupts.

There were no errors listed during transcoding.

[nyanmisaka]

Confirmed the VP9 ICQ is also broken (device failed -17) on DG2 even if the driver reports VA_RC_ICQ.

[janbrueggenjuergen]

Can also confirm ICQ broken for AV1.

[Sherry-Lin]

@OttoYang @bai-isaac could you take a look?

[nyanmisaka]

HEVC ICQ is now fixed in 36275b06.

AV1 and VP9 ICQ have not been fixed yet.

[bavdevc]

Hello Intel staff,

could someone at Intel please investigate on what we're waiting for? having the same ICQ features we currently have on windows platform working, also working on linux platform? should not be that difficult

please fix that or respond to your customers why you cannot do that strategically

btw. VP9 ICQ doesn't even work with Windows driver, but AV1 should seriously be fixed - Intel and AV1 are set in stone for the future

br, bavdevc

[bavdevc]

I really hope we can continue with fixing linux bugs when the Microsoft feature enablement is fixed... https://github.com/intel-media-ci/ffmpeg/pull/619 - pretty please!

[bStyler]

HEVC ICQ is now fixed in 36275b0.

AV1 and VP9 ICQ have not been fixed yet.

Hi @nyanmisaka , are there any intentions of getting VP9 encoding fixed?

[nyanmisaka]

@bStyler I'm not from intel. The community is waiting for a response from intel for the AV1 and VP9 ICQ.

[XinfengZhang]

sorry for the issue, TBH, we have no plan to enable ICQ of AV1 and VP9 on DG2 linux, because we assume there is few usage. could you share the usage or some background of this request, and let we have the opportunity to revisit the usage and correct our assumption ?

[nyanmisaka]

ICQ rate control can reduce the file size while maintaining high quality, which is what VBR and CBR are not good at. These two are mainly used for live streaming. Users usually use ICQ to archive movies to save hard drive space. Common uses are ffmpeg, Handbrake, Tdarr (Distributed Transcoding System), etc.

Also, in fact, with the existence of AV1, VP9 is not so important. It is more practical to implement ICQ in AV1 encoder since it's a killer feature in Arc discrete graphics.

In addition, Linux is widely used in servers, including but not limited to small home servers. If AV1 ICQ is only available on Windows, the user has to spend time passing through the graphics card to the Windows VM, which also adds to the unreliability.

could you share the usage or some background of this request, and let we have the opportunity to revisit the usage and correct our assumption ?

Any more comments from you guys? @channeladam @vid-bin @janbrueggenjuergen @bavdevc @bStyler

[channeladam]

I agree AV1 ICQ is much more important than VP9 ICQ, since AV1 supersedes it.

As @nyanmisaka said, the benefit of ICQ is the smaller file size with the similar quality. And Linux is widely used in servers.

I would add that it is not just 'home servers' and 'movies' though. Many businesses run Linux workstations (or would prefer to) for their content creation and local video processing.

Not having the better AV1 ICQ capability on Linux is detrimental to (a) those businesses (as it incurs higher costs by being forced to use alternatives) and (b) the Open Source community.

And it hurts Intel's brand reputation. I purchased the Intel Arc A770 for what I understood is its better encoding quality. Supporting ICQ on Windows and not Linux is a kick in the teeth to me from Intel, and makes me reconsider any further Intel investment.

[bavdevc]

Yes - the Intel ICQ mode is the recommend option in Intel encoder technologies when you want to achieve something like the widely used CRF mode in H264/H265 software encoders. There is definitely a serious use case for that (official doc from Intel):

Intelligent Constant Quality (ICQ) Rate Control

The ICQ bitrate control algorithm is designed to improve subjective video quality of an encoded stream: it may or may not improve video quality objectively - depending on the content. ICQQuality is a control parameter which defines the quality factor for this method. ICQQuality parameter can be changed between 1 - 51, where 1 corresponds to the best quality. The achieved bitrate and encoder quality (PSNR) can be adjusted by increasing or decreasing ICQQuality parameter. This rate control is recommended for storage solutions, where high quality is required while maintaining a smaller file size.

Please enable that for AV1/linux at least - you can see some of the referenced requests from all different projects that use intel libraries in this issue #1597 and you can surely find some requests in your internal support/help desk.

edit: there are no "storage transcode/solutions" available for discrete graphics/Arc A-Series if this bug isn't fixed.

LA / LA-HRD isn't working on fixed-function HW - ICQ is the only option available on that hardware (atm only when using Windows OS - Linux support is missing).

btw. onevpl spec clearly tells us that Intel Arc A-Series supports AV1 - Specialized BRC modes - ICQ and there is no restriction for "(Windows Only)" like you see it with other features https://www.intel.com/content/www/us/en/docs/onevpl/developer-reference-media-intel-hardware/1-0/features-and-formats.html#ENCODE-DISCRETE

[thiagorb]

@XinfengZhang

Many businesses run Linux workstations (or would prefer to) for their content creation and local video processing.

This is also the case for us. We would like to use our Linux servers to transcode video files from users, and using using hardware acceleration is a must have.

[vid-bin]

sorry for the issue, TBH, we have no plan to enable ICQ of AV1 and VP9 on DG2 linux, because we assume there is few usage. could you share the usage or some background of this request, and let we have the opportunity to revisit the usage and correct our assumption ?

Will intel be offering a full refund for users who bought their arc cards specifically for this purpose then? I mean I went with one of the cheapest ones but I'd still like my $$$ back in this scenario. The card is going to be a paper weight for me without CRF/ICQ options.

[bavdevc]

@nyanmisaka your enablement for the HEVC/ICQ code #https://github.com/intel/media-driver/commit/36275b060c261cd902023e01d12ed6561f6b5a58 works brilliantly, thank you very much - I appreciate it alot

[janbrueggenjuergen]

I am a master IT student and intended to use an arc 380 card in my home server for transcoding and archival of multimedia for my hosted jellyfin server. For the time beeing, not incorperating linux support is a huge down for intel in my eyes as it contradicts the established image of not so closed eco systems like nvidia and actually forces me probably to switch back when rtx 3000 cards become cheaper.

[lol]

sorry for the issue, TBH, we have no plan to enable ICQ of AV1 and VP9 on DG2 linux, because we assume there is few usage. could you share the usage or some background of this request, and let we have the opportunity to revisit the usage and correct our assumption ?

Assuming very few will use ICQ with AV1 on Linux is the wrong approach.

The entire reason why a lot of people bought DG2 is for transcoding. Specifically, to use with their Jellyfin, Emby and Plex media servers.

I hope this issue gets priority. If this works, expect Intel graphic cards to absolutely dominate the sales for this use case.

[Enverex]

could you share the usage or some background of this request

... to use the hardware we purchased to the best of it's ability? Why would we want to use inferior encoding modes? This seems like a weird question. If ICQ encodes result in better quality at the same bitrate, or lower bitrate at the same quality, why would you think that people would be happy with the lesser options?

[XinfengZhang]

thanks a lot for your input, thanks @channeladam @nyanmisaka @bavdevc @thiagorb @vid-bin @janbrueggenjuergen @lol @Enverex , we revisited the assumption with your helps. and We will enable ICQ mode, the patch will be ready in few weeks.

[bavdevc]

@XinfengZhang thanks a lot for that positive feedback - and while implementing it for linux please make sure that it works the same as the windows version - there are some differences with HEVC ICQ mode already between Windows/Linux, please note that people are using that Intelligent Constant Quality (ICQ) Rate Control in many cross platform software projects (edit: that means the same ICQ value should produce the same quality&filesize on Windows and Linux - identical ICQFactorLookup and such things, please - https://github.com/oneapi-src/oneVPL-intel-gpu/commit/044994c38525e3016b0c9019f122dfde76fb01e6)

[BlakeB415]

Any progress updates on this? This is a highly wanted feature for archival.

Will there also be support for QVBR once this is implemented?

[janbrueggenjuergen]

Any progress updates on this? This is a highly wanted feature for archival.

Considering a quarterly release cycle, i wouldn't expect change before mid of august.

[vid-bin]

I see this is now in progress. Is this just for AV1 or will VP9 also receive ICQ support?

(I really just care about AV1 though)

[BlakeB415]

Hey, I noticed this had been placed in progress on July 20th. It's currently August 17th. Do you have any progress updates you can share?

Thank you.

[leyu-yao]

We are now working on final quality tuning. The patch will be online soon.

[vid-bin]

I noticed it was downgraded to priority 2. Does this mean it's ready for release?

[nyanmisaka]

9ffcdf4

[leyu-yao]

Thanks all of you. We have upstream ICQ implementations in media-driver and oneVPL-intel-gpu Media-driver commit https://github.com/intel/media-driver/commit/9ffcdf4d3d2d81b98a0d3a89a39afcc3388bf2a2 oneVPL commit https://github.com/oneapi-src/oneVPL-intel-gpu/commit/4157fdbd56bbd608bbe119ac0cc72683d782cd12

I'm closing this issue now. Please feel free to re-open for any other concerns or issues.

[vid-bin]

It appears to work from initial testing.

ffmpeg does throw this error with a simple command:

[av1_qsv @ 0x55dc9d895880] Warning in encoder initialization: incompatible video parameters (5)

ffmpeg -i "test.webm" -c:v av1_qsv -global_quality 20 -pix_fmt p010le -c:a libfdk_aac -profile 1 "output.mp4"

[xhaihao]

It appears to work from initial testing.

ffmpeg does throw this error with a simple command:

[av1_qsv @ 0x55dc9d895880] Warning in encoder initialization: incompatible video parameters (5)

av1_qsv works for me without this warning.

ffmpeg -i "test.webm" -c:v av1_qsv -global_quality 20 -pix_fmt p010le -c:a libfdk_aac -profile 1 "output.mp4"

[leyu-yao]

Hi @vid-bin This warning looks like not related to ICQ itself. I'm suspecting it as a tile setting issue. It is very likely that this is a VPL issue.

[BlakeB415]

Hi, QVBR doesn't seem to be working with this. ffmpeg -i input.mp4 -c:v av1_qsv -global_quality:v 25 -b:v 1000k -maxrate:v 1001k output.mp4 results in

[av1_qsv @ 0x561f1699f4c0] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
[vost#0:0/av1_qsv @ 0x559966baf140] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height

[vid-bin]

@BlakeB415

Doesn't seem to be implemented yet.

[av1_qsv @ 0x55f1cd368900] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
[vost#0:0/av1_qsv @ 0x55f1cd0a9980] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
Error while filtering: Function not implemented

[BlakeB415]

@BlakeB415

Doesn't seem to be implemented yet.

[av1_qsv @ 0x55f1cd368900] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
[vost#0:0/av1_qsv @ 0x55f1cd0a9980] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
Error while filtering: Function not implemented

I see. Well, this works with h264_qsv on Linux so I was hoping this would add support for av1_qsv for it as well.

EDIT: There seems to be references to QVBR within the media-driver code for AV1 so maybe this is an issue upstream?

[leyu-yao]

Hi @BlakeB415 Unlike avc, av1 does not supported QVBR, at least for now. There is something common in ICQ and QVBR, so it is expected that some ICQ code may "look like" QVBR. I don't think this is an issue.

[Feierkuula]

I tested it, it works perfectly.

[Enverex]

Does this not include the LookAhead functionality? (i.e. icq_la).