Enable Qlora scripts on Intel GPUs

artidoro / qlora

QLoRA: Efficient Finetuning of Quantized LLMs

https://arxiv.org/abs/2305.14314

MIT License

9.96k stars 820 forks source link

Enable Qlora scripts on Intel GPUs #219

Closed abhilash1910 closed 1 year ago

abhilash1910 commented 1 year ago

Motivation:

Thanks for the qlora repo, this PR is for adding intel gpu support for quantized inference in the qlora script.
There is an internal progress on B&B enabling for intel gpus which will make the support complete.

@artidoro could you please review . Would appreciate your thoughts on this. Thanks

abhilash1910 commented 1 year ago

@artidoro requesting re-review , let me know if any changes needed else can this be merged?

artidoro commented 1 year ago

So it looks good to me, however, I cannot test the code as I don't have an Intel GPU. Could you let me know how you verified that it works? @TimDettmers do you have thoughts on adding this to the repo? It could be helpful to people using Intel GPUs and it seems relatively few changes.

abhilash1910 commented 1 year ago

@artidoro @TimDettmers at Intel we are planning to integrate our kernel code for the quantization cu kernels . This is a work in progress internally and after validation I will flag a PR at b&b. The pythonic frontend (here) has relatively easier changes but the core of the task is in the kernel adaptation for our gpu devices which we are integrating.

artidoro commented 1 year ago

Sounds good! I am happy to merge this PR. Might be useful to add some documentation too.