Failed building wheel for auto-gptq

[pyrater]

` "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\bin\nvcc" -c autogptq_cuda/autogptq_cuda_kernel.cu -o build\temp.win-amd64-cpython-310\Release\autogptq_cuda/autogptq_cuda_kernel.obj -IC:\Users\jsviv\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\include -IC:\Users\jsviv\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\jsviv\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\include\TH -IC:\Users\jsviv\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\include" -Iautogptq_cuda -IC:\Users\jsviv\AppData\Local\Programs\Python\Python310\include -IC:\Users\jsviv\AppData\Local\Programs\Python\Python310\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.36.32532\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22000.0\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=autogptq_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --use-local-env C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\include\crt/host_config.h(160): fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2019 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk. autogptq_cuda_kernel.cu error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\bin\nvcc.exe' failed with exit code 2 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for auto-gptq Running setup.py clean for auto-gptq Failed to build auto-gptq ERROR: Could not build wheels for auto-gptq, which is required to install pyproject.toml-based `projects``

Have NVIDIA card, NVIDIA Toolkit mapped to environmental var, and visual studio 2019, Still wont work.

[marella]

In the error it shows it is trying to use C:\Program Files (x86)\Microsoft Visual Studio\2022 not 2019. Can you please try updating CUDA to 11.8 Also the CUDA toolkit version should be same as the CUDA version PyTorch was compiled with for auto-gptq to work. Can you please share the PyTorch version you have installed (pip freeze). It looks something like torch==2.0.1+cu118

If you have CUDA 11.8 you can also download and install prebuilt wheels from here that don't require compilation:

pip install auto_gptq-0.2.2+cu118-cp310-cp310-win_amd64.whl

If none of these work, you can disable building the CUDA extension:

BUILD_CUDA_EXT=0 pip install auto-gptq

[pyrater]

pip install auto_gptq-0.2.2+cu118-cp310-cp310-win_amd64.whl fixed it.

[pyrater]

Thank you!