search

HandH1998 / QQQ

QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
https://arxiv.org/pdf/2406.09904
91 stars 8 forks source link

Question about building W4A8 on AMD platform #22

Closed XIAOHUIL1 closed 1 month ago

XIAOHUIL1 commented 1 month ago

First of all, thank you for your excellent work. We have successfully reproduced your work on Nvidia-A800. However, I don't know if this is also supported on the AMD platform. I hope to get an answer. Thanks again~

HandH1998 commented 1 month ago

I think the answer is probably no. I developed and tested the w4a8 GEMM only on Nvidia GPU sm > 8.0.

XIAOHUIL1 commented 1 month ago

I think the answer is probably no. I developed and tested the w4a8 GEMM only on Nvidia GPU sm > 8.0.

This is a pity, but thank you for your outstanding contribution.