Closed qy1026 closed 4 months ago
Single gpu(48GB) can't support qwen1.5-72B-awq-int4 to be converted to .pt format, which will trigger an OOM error. How can I use multi gpus in qwen2/quantize.py?
unmark the line 113 and 114 may work it. link
already fix
Single gpu(48GB) can't support qwen1.5-72B-awq-int4 to be converted to .pt format, which will trigger an OOM error. How can I use multi gpus in qwen2/quantize.py?