search

artidoro / qlora

QLoRA: Efficient Finetuning of Quantized LLMs
https://arxiv.org/abs/2305.14314
MIT License
9.92k stars 817 forks source link

AMD support #185

Open Amshaker opened 1 year ago

Amshaker commented 1 year ago

Hi authors,

Thank you for your awesome work.

I have a question, please. Is qlora supports AMD GPUs? I don't have CUDA and I have errors regarding quantization "undefined symbol: cquantize_blockwise_fp16_nf4". I am worry this error because of the GPU. Please let me know your thoughts.

Best regards, Abdelrahman.

ehartford commented 10 months ago

hello I have AMD too I would like to use qlora

(textgen) eric@quixi1:~/text-generation-webui$ lspci | egrep -i "display|vga"
03:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
23:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
26:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
43:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
64:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41)
83:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
a3:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
c3:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
c6:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)