cuda 12.2 AssertionError: AWQ kernels could not be loaded.

casper-hansen / AutoAWQ_kernels

MIT License

48 stars 19 forks source link

cuda 12.2 AssertionError: AWQ kernels could not be loaded. #16

Open s-natsubori opened 5 months ago

s-natsubori commented 5 months ago

I am facing error, AWQ kernels could not be loaded.with autoawq==0.2.4.

image nvcr.io/nvidia/pytorch:23.10-py3
- Python 3.10
- cuda 12.2.2
- torch 2.1.0a0+32f93b1

Build and install from source, it works. Are there any plans to provide build packages for cuda12.2 ?

abduIbasit commented 2 months ago

i have cuda version 12.4 and unable to downgrade to 12.2, building and installing from source doesn't fix the issue for me @casper-hansen could you please look at the issue with the kernels not being able to be loaded

wasertech commented 1 month ago

I encountered a similar issue with CUDA 12.5. The problem was that the PyTorch version pinned in setup.py was prebuilt with CUDA 12.2. To resolve this, I created a branch (#29) that allows building the kernel from source with CUDA 12.5 (or any other for that matter). I’ve successfully built the kernel multiple times using this branch. Feel free to check it out and let me know if you have any questions!