Support for AMD ROCM - Githubissues

turboderp / exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

MIT License

2.66k stars 214 forks source link

Open yehowshuaradialrad opened 10 months ago

yehowshuaradialrad commented 10 months ago

I have a machine with Mi25 GPUs. Would anybody like SSH access to develop on it for exllama?

TNT3530 commented 10 months ago

Exllama already works with ROCm, just not at full performance