CUDA extension not installed

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

MIT License

626 stars 49 forks source link

CUDA extension not installed #62

Closed Arthur-Ling closed 2 months ago

Arthur-Ling commented 5 months ago

When I try to run scripts, "CUDA EXTENSION NOT INSTALLED" happens, and the running time is too long. What should I do?

Arthur-Ling commented 5 months ago

And when i run: " CUDA_VISIBLE_DEVICES=6 python main.py \ --model facebook/opt-6.7b --eval_ppl \ --epochs 0 --output_dir ./log/opt-6.7b-w4a4 \ --wbits 4 --abits 4 --lwc --let --alpha 0.75 \ --net opt-6.7b \ --resume /home/Project/OmniQuant/pre_train/opt-6.7b-w4a4.pth "

AttributeError: 'QuantOPTDecoderLayer' object has no attribute 'input_layernorm'

What should I do?

ChenMnZ commented 4 months ago

I have fixed the bug about opt models, please try again.