search

johnsmith0031 / alpaca_lora_4bit

MIT License
534 stars 84 forks source link

ModuleNotFoundError: No module named 'quant_cuda' #33

Closed winglian closed 1 year ago

winglian commented 1 year ago

https://github.com/johnsmith0031/alpaca_lora_4bit/blob/234004ceb5135e092bc9a08a9dbb75eff61f8fd9/matmul_utils_4bit.py#L3

looks like this addition requires some additional documentation or setup to get quant_cuda as this wasn't required before.

johnsmith0031 commented 1 year ago

Yes, quant_cuda extension is needed. you should run pip install -r requirements.txt to install it. If it cannot be installed, first run pip uninstall gptq_llama and reinstall it.

winglian commented 1 year ago

looks like an explicit import is needed. https://github.com/johnsmith0031/alpaca_lora_4bit/pull/34