Can't install AutoAWQ 0.2 on Windows

[kirayomato]

I downloaded autoawq-0.2.2+cu118-cp310-cp310-win_amd64.whl in releases but failed to install it. The error message is as follows。

ERROR: autoawq_kernels-0.0.5+cu118-cp310-cp310-windows_amd64.whl is not a supported wheel on this platform.

Then I tried installing the historical versions and found that version 0.1.8 installs correctly, any >=0.2.0 can not be installed.

But version0.1.8also doesn't work, when I try to run it after installing it, I get the following error

D:\Program\VLM\gptq>C:/Software/Miniconda/envs/gptq/python.exe d:/Program/VLM/gptq/Quant_awq.py
Traceback (most recent call last):
  File "d:\Program\VLM\gptq\Quant_awq.py", line 1, in <module>
    from awq import AutoAWQForCausalLM
  File "C:\Software\Miniconda\envs\gptq\lib\site-packages\awq\__init__.py", line 2, in <module>
    from awq.models.auto import AutoAWQForCausalLM
  File "C:\Software\Miniconda\envs\gptq\lib\site-packages\awq\models\__init__.py", line 1, in <module>    
    from .mpt import MptAWQForCausalLM
  File "C:\Software\Miniconda\envs\gptq\lib\site-packages\awq\models\mpt.py", line 1, in <module>
    from .base import BaseAWQForCausalLM
  File "C:\Software\Miniconda\envs\gptq\lib\site-packages\awq\models\base.py", line 12, in <module>       
    from awq.quantize.quantizer import AwqQuantizer
  File "C:\Software\Miniconda\envs\gptq\lib\site-packages\awq\quantize\quantizer.py", line 11, in <module>
    from awq.modules.linear import WQLinear_GEMM, WQLinear_GEMV
  File "C:\Software\Miniconda\envs\gptq\lib\site-packages\awq\modules\linear.py", line 4, in <module>
    import awq_inference_engine  # with CUDA kernels
ImportError: DLL load failed: The specified module could not be found.

[casper-hansen]

HI @kirayomato, could you try to install the main branch to confirm the issue is still present? Also, are you using WSL2 (a requirement)?

pip install git+https://github.com/casper-hansen/AutoAWQ.git

[kirayomato]

HI @kirayomato, could you try to install the main branch to confirm the issue is still present? Also, are you using WSL2 (a requirement)?

pip install git+https://github.com/casper-hansen/AutoAWQ.git

Still unable to install. I'm not using wsl2.

D:\Program\VLM\gptq>pip install git+https://github.com/casper-hansen/AutoAWQ.git
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting git+https://github.com/casper-hansen/AutoAWQ.git
  Cloning https://github.com/casper-hansen/AutoAWQ.git to c:\users\admin\appdata\local\temp\pip-req-build-7db2ri3w
  Running command git clone --filter=blob:none --quiet https://github.com/casper-hansen/AutoAWQ.git 'C:\Users\admin\AppData\Local\Temp\pip-req-build-7db2ri3w'
  Resolved https://github.com/casper-hansen/AutoAWQ.git to commit 68c727a1a338a1e8d988e8f6094e0d38040e0bb6
  Preparing metadata (setup.py) ... done
ERROR: autoawq_kernels-0.0.6+cu118-cp310-cp310-windows_amd64.whl is not a supported wheel on this platform.

[casper-hansen]

I'm not sure how to solve this issue unfortunately. I do not have a Windows PC at my disposal and it used to work. Open to contributions if you figure out what the issue is.

[DeepBlue27]

hello @casper-hansen I've been having this issue too, from what i've seen in your "kernel" repository the win64 wheel might be differently named, do you think that might be the problem?

[oobabooga]

I'm monitoring this issue as it prevents me from updating AutoAWQ in my project.

[casper-hansen]

Hi @oobabooga, I would love to resolve this issue but I have no means to reproduce it currently. The wheels build on Windows, so my immediate assumption is that it could be an environment/dependency issue

[Malrama]

@oobabooga @casper-hansen Triton is a dependency of AutoAWQ since https://github.com/casper-hansen/AutoAWQ/commit/8849043a0d2661ff178df6d8028114c1980328e3. There are no Triton builds for Windows, thus the dependency can't be resolved. Triton is Linux only https://pypi.org/project/triton/#files Either make Triton an optional dependency or this project can't be installed on Windows anymore.

[casper-hansen]

Triton has been removed in 0.2.3 as I realized it was not needed anyways.

[oobabooga]

I assume it should work now then. https://github.com/casper-hansen/AutoAWQ/pull/365/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7L92

[casper-hansen]

@kirayomato it seems this issue relates to not using WSL2 as far as I can see. As noted in the AutoAWQ-kernels repository, WSL2 is a requirement for Windows users.

Please refer to Microsoft's documentation and let me know if you face any further issues on your Windows machine. https://learn.microsoft.com/en-us/windows/wsl/install

[kirayomato]

@kirayomato it seems this issue relates to not using WSL2 as far as I can see. As noted in the AutoAWQ-kernels repository, WSL2 is a requirement for Windows users.

Please refer to Microsoft's documentation and let me know if you face any further issues on your Windows machine. https://learn.microsoft.com/en-us/windows/wsl/install

So this repository actually not support Windows? WSL2 is Linux.

[casper-hansen]

It doesn’t support pure Windows as Microsoft has trouble supporting CUDA code. Not much I can do about it

[suparious]

Although possible, it is very difficult to get CUDA working with python bindings natively in windows. NVIDIA has some instructions here: https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html

• this also has instructions on compiling your own wheels

But, I think currently, the only supported method is to use WSL2 (Linux).

[suparious]

WSL2 is Linux.

It is slightly different and requires its own binaries when compared to native Linux. That is why WSL2 is supported as a separate platform from Linux.