not support evaluation with ROCM

jy-yuan / KIVI

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

https://arxiv.org/abs/2402.02750

MIT License

121 stars 10 forks source link

not support evaluation with ROCM #10

Open ym-guan opened 2 weeks ago

ym-guan commented 2 weeks ago

I found that it is not support for the follow command line "cd quant && pip install -e ." with ROCM because of the cpp_extention.cpp May I pull request to solve this problem?

jy-yuan commented 2 weeks ago

Sure! We welcome contributions from the research community to improve KIVI.