Hardware decode "--avhw" not working with 10-Bit AV1

[Vrael0n]

Using "--avhw" on a 10-Bit AV1 source fails to decode while "--avsw" works. Running "VCEEncC64.exe --check-features" says the GPU is 10-Bit AV1 Encode capable, but not 10-Bit AV1 Decode capable even though the hardware can decode it.

Error:

avvce: codec av1(yuv420p10le) unable to decode by vce.

I can hardware decode this 10-Bit AV1 video in both MPV and FFmpeg:

ffmpeg.exe -benchmark -hwaccel d3d11va -i "C:\Users\SYS3\Desktop\Encode\MW2 5s clip.mkv" -f null -

Is it possible this is due to you using an old libdav1d version? I see libdav1d show up whenever I use AV1 hardware decoders (AMD/NV/INT). I see you use 1.0.0 while FFmpeg and MPV use 1.2.1

Environment Info: VCEEnc 8.16 (x64) OS Windows 11 x64 (22621) [UTF-8] CPU 12th Gen Intel Core i3-12100F [4.09GHz] (4C/8T) GPU Radeon RX 7600 [ID:0x7480], driver version: 31.0.21023.2010

Command: VCEEncC64.exe --avhw --codec av1 --preset slower --vbr 100000 --profile main --output-depth 10 --colorrange auto --colormatrix auto --colorprim auto --transfer auto --chromaloc auto --max-cll copy --master-display copy --audio-copy --log log.txt --log-level debug -i "C:\Users\SYS3\Desktop\Encode\MW2 5s clip.mkv" -o "C:\Users\SYS3\Desktop\Encode\MW2 5s clip VCE.mkv"

Source video: https://drive.google.com/file/d/101RBaA92B9APdI8RD2WIUjv_ORnrVAzH/view?usp=sharing

Debug log: log.txt

[DimkaTsv]

Ngl, when i try to open it with my 6750XT, in MPC-HC this video stutters hellishly with D3D11/D3D11 mode, plays first half smoothly and second half becomes latent and stuttery with D3D11/D3D9 mode. When i try to open video through Windows 11 "Photos" app it plays better than with MPC-HC, but still second half begins to show infinite latency increase and stuttering.

It does use HW decoding though, just performance is terrible, even though 4k60 should be doable (8k30 is limit). Reencoded to HEVC10 though works fine. Maybe i should ask support if there is heavier limitations for AV1 decoder for RDNA2? Or! Maybe such limitation comes from fact that your source is AV1 10bit with HDR encoded metadata? These ones are MUCH heavier on coder than common varieties. ... Right... I remember now having much better time with 8k30 AV1 source (aka everything was fine), so i am pretty sure that your issue lies specifically in fact that your sample is 10bit and HDR one as even 4k60 gives a lot of issues.

Hmm... And VCEEncC authomatically selects avsw , which will decode things correctly. Are you sure you need to force --avhw, as autoselection doesn't even consider it? You usually don't need to change decoders if everything works fine, you know...

Anyways, i don't think this particular problem is much of an VCEEncC issue. It gives you specific error for your config and selects suitable decoder by default. But if this can be improved is different question, maybe it can, maybe it cannot, only Rigaya can answer, as i am not very proficient in hw decoder options here. But DXVA checker reports no specific profile for AV1 10bit decoder.

[Vrael0n]

The stuttering playback is because AMD has the least performant AV1 hardware decoder and this video is ~460Mbps. It's not the resolution, frame rate, or HDR. I have tested this video on an RX 7600 (24 fps), RTX 4090 (64 fps), and 13 gen i5 iGPU (73 fps).

Auto selection may not be selecting avhw because VCEEnc doesn't know the hardware supports it, but other software is clearly using the GPU's hardware decode engine while the system's quad core is barely being touched at 4% usage.

I think it could be an old library in VCEEnc that doesn't have the feature enabled in its version.

[DimkaTsv]

Tested myself (and i now must save this command)... I do get about 21 fps only for this as result. I know AMD decoder is quite weaker than Nvidia one, as already checked data on both. RDNA 2 have 8k30 limit while RTX 3000 had 8k60. And i am pretty sure than Nvidia encoder also kinda scales with chip, meaning RTX 4090 will have more capabilities than 4060. At least in theory, they still should follow spec.

But 460 Mbps for such video is kinda too much don't you think? If it is bitrate that tanks performance.

[Vrael0n]

The decode performance at 24 fps is fine for this use if the encoder can only do about 24 fps. This issue was opened because hardware decode is not working in VCEEnc.

[NeedsMoar]

The first AV1 level that's able to decode 460Mb/s (just barely) is level 6.1 high which is made for 16k video but can also support 60fps 8k... RDNA3 supports that normally, but those 7600s are the lowest priced budget version of the card, some stuff may have been stripped out or ASICs re-used from 6000 series under the (probably correct) assumption that nobody is going to be trying to use them to watch 4k content (specifically from youtube since the good streaming sites all serve H.265 which decodes fine on the RDNA2 decoder at any commercially available bitrate including UHD BD and higher). I'd guess that AMD's hardware decoder only supports main tier because AV1 at 8k is practically non-existent and those 240Mb / 480Mb levels just aren't needed for 4k or slower 8k. AV1 isn't seekable and is too slow to be a format used by anyone as an editing source so they don't need to worry about that (and they don't directly support playback of all-intra HEVC or other formats cameras actually shoot in anyway, although it isn't too hard to decode the intra frames as a sequence for an editor).

I'm not a fan of AV1 anyway. Google has created this big stink about H.265 have a confusing licensing situation or somesuch garbage, but it's pretty clearly spelled out on the MPEG-LA website that normal end users don't need to pay anything, if you're a non-business or make under something like $100k a year off of a website serving video in the format you don't need to pay anything, if you're writing open source software you don't have to pay anything, and the decoder license for the GPU was a whopping 20 cents. So why doesn't Google like it? They made youtube into a subscription service in addition to the ad revenue to capitalize further off of ad hits on all the dopes they've managed to trick into thinking they'll become famous as an "influencer". They have a lot of these people but not much income from each.

Subscription services are the ones that get hit hardest by licensing fees on H.265, something like 25 cents per video with no upper limit, possibly with an additional per-subscriber fee (all the other uses that require royalties top out at less than it costs a west coast tech company to hire a single halfway skilled developer). That isn't a big hit for Netflix / Amazon / Hulu who don't stream tons of their shows as H.265 to begin with, saving it for their original series with HDR10 and / or DolbyVision or HDR10+ and which can predict the amount of content they'll have each year in that format, but YouTube basically operates on the late 90s herbal viagra e-mail spammer model except with users crapflooding the site with clickbait videos to try to suck people in. Since they have subscribers at all, if they didn't have a viable alternative to H.265 they'd be paying every time somebody uploaded a video in HDR, their profit margins would collapse, and youtube might actually go back to not sucking again because it would just be people uploading things they made because they wanted to rather than trying to be a celebrity.

Add to that the fact that they have a monstrous farm of TPUs, GPUs, and CPUs for other purposes that isn't in use all the time so the hardware costs are already paid and software that can distribute encoding of a single video across everything and somebody probably sat down and worked out that they're paying less in electricity over time than they would with the yearly license fees and they used more of their resources to come up with a codec they didn't have to pay for. I don't have a problem with any of that. I have a problem with the fact that they've tried and succeeded in convincing tons of people that AV1 is ever worth touching for home use by implying that they might kinda maybe owe somebody money if they use H.265 instead because it's all so confusing when the reality is that it's been spelled out clearly for years...

[Vrael0n]

I use AV1 because it achieves a higher visual quality to bitrate ratio than HEVC, nothing licensing related. I don't see how a 700 word mostly unrelated anti AV1 rant will solve this issue. 🙄

[DimkaTsv]

I use AV1 because it achieves a higher visual quality to bitrate ratio than HEVC, nothing licensing related. I don't see how a 700 word mostly unrelated anti AV1 rant will solve this issue. 🙄

At this point you could as well just use almost loseless recording quality via CQP... Which is about 300 mbps for 1080p60 (==1200 mbps for 4k60). Like reduce quality by few % and you can fit 4k60 stream within these 450 mbps even without AV1 with basically loseless quality (aka no visual differences).

Meaning, your 460 mbps AV1 10bit recordings are mostly waste of memory. You are FAR beyond efficiency peak for recording. Btw, AV1 compression efficiency matters ONLY on low bitrates, aka 6000 kbps, for example. Which is point of creation of better compression algorithms (aka to REDUCE space taken by video without sacrificing too much of a quality). With your extremely bloated channel differences between HEVC and AV1 are insignificant. Moreover argument can be made that more efficient at compression algorithm can preserve less details at bloated bitrate just because it compresses stuff more and just fills data to keep bitrate steady.

[rigaya]

I don't see a way to work through this crush when I use hw decoder.

Therefore, I have disabled hw decoder for AV1 10bit decoding in VCEEnc 8.17 to avoid the crush, AV1 10bit decode will work fine through software decoder.