search

Nevcairiel / LAVFilters

LAV Filters - Open-Source DirectShow Media Splitter and Decoders
GNU General Public License v2.0
7.23k stars 786 forks source link

Missing 4:2:2/4:4:4 DXVA hwaccel support on Intel graphics #547

Open nyanmisaka opened 1 year ago

nyanmisaka commented 1 year ago

First of all I want to thank you for this great project, I'm enjoying it so much on my Windows machine.

Intel graphics have supported decoding of HEVC 4:2:2 and 4:4:4 8/10bits since Ice Lake released in 2019. But even in 2023, Microsoft has not provided any official header file and struct for the HEVC Rext. Fortunately Intel made it possible for Chromium browser and a few video players such as VLC and MPC-BE to leverage it by using the vendor-specific DXVA entries.

It would be great if we can also have this feature in LAVFilters, given that 4:2:2 is becoming more and more common in the output format of high-end SLR cameras. It's difficult for users to play them on Windows smoothly without a powerful CPU, especially when it comes to 8k 4:2:2 clips.

I read this discussion from three years ago, can we revisit the patch series? Friendly paging @Nevcairiel https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200313102354.2500-1-robux4@ycbcr.xyz/

Screenshot 2023-07-03 225454

Nevcairiel commented 1 year ago

If the spec becomes official, I'll gladly support it. Until then, not adding code without a spec reference.

nyanmisaka commented 1 year ago

Intel media developer recently pointed out in Chromium-Review that Microsoft will soon release the DXVA Rext specification, we just have to wait.

Nice to see that. They almost forgot about it.

Seems like my tiny effort is paying off :P https://github.com/intel-media-ci/ffmpeg/pull/602#issuecomment-1363673893

nyanmisaka commented 1 year ago

Intel also supports HEVC SCC, but not sure if Microsoft will also add support for this profile, a certain Intel media developer told me that at least for now there are no plans.

I don't see an urgent need for HEVC SCC hwaccel, and it wouldn't be a problem if it came later.

nyanmisaka commented 1 year ago

If the spec becomes official, I'll gladly support it. Until then, not adding code without a spec reference.

Thanks for the response!

Andarwinux commented 7 months ago

Microsoft has released the DXVA Rext specification. The relevant GUIDs can be found in the latest Canary SDK.

DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME,   0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12,     0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN_444,   0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN16,     0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);
nyanmisaka commented 7 months ago

Microsoft has released the DXVA Rext specification. The relevant GUIDs can be found in the latest Canary SDK.

DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME, 0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12, 0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN_444, 0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D12_VIDEO_DECODE_PROFILE_HEVC_MAIN16, 0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME,   0x0685b993, 0x3d8c, 0x43a0, 0x8b, 0x28, 0xd7, 0x4c, 0x2d, 0x68, 0x99, 0xa4);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MONOCHROME10, 0x142a1d0f, 0x69dd, 0x4ec9, 0x85, 0x91, 0xb1, 0x2f, 0xfc, 0xb9, 0x1a, 0x29);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12,     0x1a72925f, 0x0c2c, 0x4f15, 0x96, 0xfb, 0xb1, 0x7d, 0x14, 0x73, 0x60, 0x3f);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_422, 0x0bac4fe5, 0x1532, 0x4429, 0xa8, 0x54, 0xf8, 0x4d, 0xe0, 0x49, 0x53, 0xdb);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_422, 0x55bcac81, 0xf311, 0x4093, 0xa7, 0xd0, 0x1c, 0xbc, 0x0b, 0x84, 0x9b, 0xee);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN_444,   0x4008018f, 0xf537, 0x4b36, 0x98, 0xcf, 0x61, 0xaf, 0x8a, 0x2c, 0x1a, 0x33);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_EXT, 0x9cc55490, 0xe37c, 0x4932, 0x86, 0x84, 0x49, 0x20, 0xf9, 0xf6, 0x40, 0x9c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN10_444, 0x0dabeffa, 0x4458, 0x4602, 0xbc, 0x03, 0x07, 0x95, 0x65, 0x9d, 0x61, 0x7c);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN12_444, 0x9798634d, 0xfe9d, 0x48e5, 0xb4, 0xda, 0xdb, 0xec, 0x45, 0xb3, 0xdf, 0x01);
DEFINE_GUID(D3D11_DECODER_PROFILE_HEVC_VLD_MAIN16,     0xa4fbdbb0, 0xa113, 0x482b, 0xa2, 0x32, 0x63, 0x5c, 0xc0, 0x69, 0x7f, 0x6d);

@Andarwinux Great! Thanks for the heads up. cc @Nevcairiel

Nevcairiel commented 7 months ago

If there is an actual specification, a link to that would be helpful. On a quick glance I couldn't find it, but I didn't have much time to go digging right now.

Andarwinux commented 7 months ago

The online specification documentation is not yet available. Microsoft will update the stable Windows SDK shortly, and the documentation will be updated at that time. But for now the new headers needed for Rext support are available from the Canary Windows SDK, and Chromium developers will be writing a new implementation of Chromium D3D12 HEVC Rext next week.

Andarwinux commented 7 months ago

Update: DirectX-Headers already contain new redistributable headers. https://github.com/microsoft/DirectX-Headers/blob/main/include/directx/d3d12video.h#L6304

kasper93 commented 7 months ago

Apparently they ship preview SDK on nuget https://www.nuget.org/packages/Microsoft.Windows.SDK.CPP.x64/10.0.25936-preview (includes those new defines), but I feel like there is no point in doing anything until there is proper release and documentation.

Nevcairiel commented 4 months ago

At least the latest preview SDK has all the required information - however no drivers exist yet that support it, at least for non-insider windows (and I'm not bothering with that). So it might only become available once the next major update of Windows 11 rolls out.

nyanmisaka commented 4 months ago

At least the latest preview SDK has all the required information - however no drivers exist yet that support it, at least for non-insider windows (and I'm not bothering with that). So it might only become available once the next major update of Windows 11 rolls out.

Perhaps we add support by first borrowing the GUID already implemented in Intel's existing driver, and then switch when the new driver & GUIDs becomes available in Windows 11? Driver support for some early Rext-enabled Intel graphics cards such as 10th Gen Ice Lake and Jasper Lake has been moved to maintenance-only status. They may never receive drivers containing these new GUIDs, but their Intel-specific GUIDs are always available.

https://github.com/hughbe/windows-sdk-headers/blob/89f151d343587fcd5b854a4851fcb8f7187f3ac4/Microsoft.Windows.SDK.CPP/latest/Include/um/dxva.h#L1273-L1304

Still, I haven't carefully compared the differences between the old and new DXVA_PicParams_HEVC_RangeExt structures. Correct me if I'm wrong.

Nevcairiel commented 4 months ago

Unfortunately the Intel version of that structure is not compatible with the Microsoft version, so I won't be offering support for those.

nyanmisaka commented 4 months ago

Unfortunately the Intel version of that structure is not compatible with the Microsoft version, so I won't be offering support for those.

That makes sense. They are just incompatible.

SA-Root commented 1 month ago

It seems that HW decoding for HEVC 444 is now working using the D3D11 decoder in Potplayer. I'm using latest public Intel driver version 5522(https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html). So I assume the driver is no longer a problem. Hope you guys can get it working soon.

Screenshot 2024-05-28 222523
nyanmisaka commented 1 month ago

"MPC Video Renderer" uses Intel-specific HEVC Rext GUIDs.

SA-Root commented 1 month ago

"MPC Video Renderer" uses Intel-specific HEVC Rext GUIDs.

Well ,I switched to D3D11 Video Render this time, and it seems to still be using HW decode.

Screenshot 2024-05-28 230251
nyanmisaka commented 1 month ago

PotPlayer, as a closed source software, also supported Intel-specific GUIDs not long ago.

Nevcairiel commented 1 month ago

Intel actually supports a good chunk of the new GUIDs already, but there is still no official spec on their limitations etc, which I expect will arrive with the next Windows 11 major update (24H2), which will be coming out sometime in the second half of this year.

  -> HEVC / H.265 variable-length decoder, main
  -> HEVC / H.265 variable-length decoder, main10
  -> HEVC / H.265 variable-length decoder, main10 extended
  -> HEVC / H.265 variable-length decoder, main10 422
  -> HEVC / H.265 variable-length decoder, main 444
  -> HEVC / H.265 variable-length decoder, main10 444
  -> HEVC / H.265 variable-length decoder, main12
  -> HEVC / H.265 variable-length decoder, main12 422
  -> HEVC / H.265 variable-length decoder, main12 444