Intel's XMX Engine (Tensor cores - FP16 Matrix) support

[NikosDi]

Hello. I just tried latest version of your app 20230812 using an Intel Arc A380 card and the fp16-matrix value is 0.0

It is obvious that Intel's Tensor cores aren't supported.

Is it possible to add support for Intel's XMX engine of Arc cards into your next release ?

Thank you!

[nihui]

vkpeak uses the VK_KHR_cooperative_matrix extension to implement the call to the GPU matrix operation unit According to https://vulkan.gpuinfo.org/listdevicescoverage.php?extension=VK_KHR_cooperative_matrix&platform=all There is currently no driver that implements extensions on Intel graphics cards, so we need to wait for the Intel driver to be updated before making relevant adaptations.

[NikosDi]

It seems that Intel has moved to that direction, initially for Linux: https://www.phoronix.com/news/Intel-ANV-Cooperative-Matrix

[NikosDi]

Hello.

Latest Intel driver v31.0.101.5330 for Windows has added three new Vulkan extensions:

– VK_KHR_shader_atomic_int64 – VK_KHR_cooperative_matrix – VK_EXT_host_image_copy

I think we are OK now 👍

[NikosDi]

vkpeak uses the VK_KHR_cooperative_matrix extension to implement the call to the GPU matrix operation unit According to https://vulkan.gpuinfo.org/listdevicescoverage.php?extension=VK_KHR_cooperative_matrix&platform=all There is currently no driver that implements extensions on Intel graphics cards, so we need to wait for the Intel driver to be updated before making relevant adaptations.

@nihui

Hello.

Any news regarding Intel's XMX engine support ?

Thank you.

[nihui]

https://github.com/nihui/vkpeak/pull/13

[NikosDi]

13

@nihui

UHD 730 doesn't have XMX engine inside.

It is just the driver reporting some kind of "emulation" for fp16 matrix support using iGPUs like UHD 730.

You have to try your code to an Intel GPU with real fp16-matrix support using XMX engine like desktop Arc GPUs.

If you can post somewhere the executable, I could try it with my Arc A380.

[nihui]

13

@nihui

UHD 730 doesn't have XMX engine inside.

It is just the driver reporting some kind of "emulation" for fp16 matrix support using iGPUs like UHD 730.

You have to try your code to an Intel GPU with real fp16-matrix support using XMX engine like desktop Arc GPUs.

If you can post somewhere the executable, I could try it with my Arc A380.

https://github.com/nihui/vkpeak/releases/tag/20240505

[NikosDi]

@nihui

https://github.com/nihui/vkpeak/releases/tag/20240505

The app crashes Windows after a while (about a minute) while trying to run fp16-matrix, immediately after ending fp16-vec. It just stalls at fp16-matrix and crashes after that.

My configuration: Win 11 Pro - Intel Arc A380 - Driver v5445