I encountered the following problem during the KL assessment

casper-hansen / AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

https://casper-hansen.github.io/AutoAWQ/

MIT License

1.74k stars 208 forks source link

I encountered the following problem during the KL assessment #526

Open xieziyi881 opened 4 months ago

xieziyi881 commented 4 months ago

ValueError: You are attempting to load an AWQ model with a device_map that contains a CPU or disk device. This is not supported. Please remove the CPU or disk device from the device_map.

Junsong798 commented 2 months ago

I have the same issue. Did you figure that out?

carinae commented 1 month ago

Same error for me.