search

jankais3r / LLaMA_MPS

Run LLaMA (and Stanford-Alpaca) inference on Apple Silicon GPUs.
GNU General Public License v3.0
583 stars 47 forks source link

Fine tuning LLaMA on Apple Silicon GPUs #7

Closed Gincioks closed 1 year ago

Gincioks commented 1 year ago

Hello,

I am new to the AI field and still trying to understand how things work. I was wondering if it's possible to apply this implementation in the fine-tuning process. Like: https://github.com/lxe/llama-tune or https://github.com/tloen/alpaca-lora

I would be grateful for any examples or tutorials that explain how to apply this implementation to the fine-tuning process. Thank you in advance for your help!

jankais3r commented 1 year ago

Hi, I experimented with it, but so far I am running out of memory on a 128GB machine. Training and fine tuning requires significantly more memory than inference, so I am not sure I will be able to get the memory usage low enough for Apple Silicon Macs... My hope is that tloen or lxe will eventually release their fine tuned models.

jankais3r commented 1 year ago

The latest commit contains support for running alpaca inference.