Open Oscilloscope98 opened 8 months ago
@ashokei @min-jean-cho FYI. issue on Windows
@Oscilloscope98, is the error reproducible with different input sizes (e.g., smaller input sizes)?
Hi @min-jean-cho,
The same problem happened for
import torch
import intel_extension_for_pytorch as ipex
tensor1 = torch.randn(1, 1, 1, 2).to('xpu')
tensor2 = torch.randn(1, 1, 2, 1).to('xpu')
torch.matmul(tensor1, tensor2).size()
P.S. test Driver: 31.0.101.5081, test machine Arc A730M
Working on triage.
Do you have the graphics card attached to a monitor and disable iGPU in BIOS?
Do you have the graphics card attached to a monitor and disable iGPU in BIOS?
Hi @jingxu10,
For Arc A730M, it is an NUC machine, and we did not disable iGPU in BIOS.
For Arc A750, I am not sure whether the graphics card was attached to a monitor, but the iGPU was also not disabled in BIOS.
We found an issue that the card has to be attached to a monitor and disable iGPU to get the dGPU working. We are working on triaging this issue.
@Oscilloscope98
Please use following cmd to set the GPU you want.
for example, if you want to use the second GPU: export ZE_AFFINITY_MASK=1
I followed https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_windows_gpu.html#verify-installation to reproduce this issue, after setting
set ZE_AFFINITY_MASK=0
set ONEAPI_DEVICE_SELECTOR=level_zero:1
the same error still occurs
Hi @NeoZhangJianyu,
We tried again on a Windows machine (with Intel(R) UHD Graphics 770
and Intel(R) Arc(TM) A770 Graphics
available, driver 31.0.101.5382
), it seems like set ZE_AFFINITY_MASK=1
will make all xpu device unavailable:
Hi,
I had a similar problem on my NUC as well. When you have two GPUs (I have one Iris and one Arc) you need to also specify which to choose like this: to("xpu:1")
, since in my case ARC was on index 1. Hope this helps.
This issue is root caused to GPU hardware detection logic for multi-GPU systems in oneDNN. The fix is available in oneDNN v3.4.3.
@jingxu10, @min-jean-cho, it would be awesome to have IPEX patch release with this fix.
yeah, WIP.
yeah, WIP.
hi @jingxu10 , I meet this issue on a Linux machine with Intel(R) UHD Graphics 770 and Intel(R) Arc(TM) A750 Graphics available. May I ask how this fix working on? Will it be ported to Linux solution? Thanks!
Pls try disabling iGPU as a workaround at this time. We are still working on the solution.
I have the same problem. It happened when I want to use the conv2d(). At first I think it may be something wrong with gpu. So I changed the device from 'cuda:0' to 'cpu'. Unforturnately, it still happened.
x = F.relu(self.conv1(inputs))
Traceback:
File "/home/vic/miniconda3/envs/new3.8/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 454, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: could not create a primitive
Describe the bug
Machine: Arc A730M (Also met same bug on Arc A750) OS: WIndows 11 Driver: 31.0.101.5081 (Also met same bug with version 31.0.101.5084) oneAPI: 2024.0
Code to reproduce:
test.py:
Error message:
The error is still there even if we used
set ONEAPI_DEVICE_SELECTOR=level_zero:0
to make only A730M available to the environment.sycl-ls
output on machine with A730M:Versions
PyTorch version: 2.1.0a0+cxx11.abi PyTorch CXX11 ABI: No IPEX version: 2.1.10+xpu IPEX commit: a12f9f650 Build type: Release
OS: Microsoft Windows 11 专业版 GCC version: N/A Clang version: N/A IGC version: 2024.0.0 (2024.0.0.20231017) CMake version: version 3.27.2-msvc1 Libc version: N/A
Python version: 3.9.18 (main, Sep 11 2023, 14:09:26) [MSC v.1916 64 bit (AMD64)] (64-bit runtime) Python platform: Windows-10-10.0.22621-SP0 Is XPU available: True DPCPP runtime version: N/A MKL version: N/A GPU models and configuration: [0] _DeviceProperties(name='Intel(R) Arc(TM) A730M Graphics', platform_name='Intel(R) Level-Zero', dev_type='gpu, support_fp64=0, total_memory=11934MB, max_compute_units=384, gpu_eu_count=384) [1] _DeviceProperties(name='Intel(R) Iris(R) Xe Graphics', platform_name='Intel(R) Level-Zero', dev_type='gpu, support_fp64=0, total_memory=14751MB, max_compute_units=96, gpu_eu_count=96) Intel OpenCL ICD version: N/A Level Zero version: N/A
CPU: Architecture=9 CurrentClockSpeed=2300 DeviceID=CPU0 Family=198 L2CacheSize=7680 L2CacheSpeed= Manufacturer=GenuineIntel MaxClockSpeed=2300 Name=12th Gen Intel(R) Core(TM) i7-12700H ProcessorType=3 Revision=
Versions of relevant libraries: [pip3] intel-extension-for-pytorch==2.1.10+xpu [pip3] numpy==1.26.3 [pip3] torch==2.1.0a0+cxx11.abi [pip3] torchaudio==2.1.0a0+cxx11.abi [pip3] torchvision==0.16.0a0+cxx11.abi [conda] intel-extension-for-pytorch 2.1.10+xpu pypi_0 pypi [conda] numpy 1.26.3 pypi_0 pypi [conda] torch 2.1.0a0+cxx11.abi pypi_0 pypi [conda] torchaudio 2.1.0a0+cxx11.abi pypi_0 pypi [conda] torchvision 0.16.0a0+cxx11.abi pypi_0 pypi