on peut faire l'entrainement avec GPU ?

[walidbet18]

docker run --gpus "device=1" -v /home/walid.bettahar@transvalor.com/models:/models -v /home/walid.bettahar@transvalor.com/dataset:/dataset ghcr.io/ggerganov/llama.cpp:full-cuda --finetune --model-base /models/dolphin-2.2.1-mistral-7b.Q2_K.gguf --lora-out /models/lora-open-dolphin-2.2.1-mistral-7b.Q2_K-transvalor-ITERATION.bin --train-data /dataset/output.txt --save-every 10 -b 15 --adam-iter 256 --checkpoint-in chk-dolphin-2.2.1-mistral-7b.Q2_0-trsvalor-LATEST.gguf --checkpoint-out chk-dolphin-2.2.1-mistral-7b.Q2_0-trsvalor-ITERATION.gguf --ctx 64 --no-checkpointing -ngl 13

quand j'utilise les gpu pour le fine tunning ca quitte avec une erreur 7 segmentation fault !! pourquoi ?

[JohannesGaessler]

En anglais s'il vous plaît.

[walidbet18]

can i use GPU to fine tune my model , when i use the command that i sent earlier the fine tunning stops with "7segmentation fault" error but when i run it on cpu it works and effectivly it takes long time so i'm wondering if we can do the fine tunnig in gpus , and what's the source of the problem ?

[JohannesGaessler]

To my knowledge it is currently not possible to train using GPUs and the problem is that it's simply not implemented. One of my goals for 2024 is to do this implementation.

[walidbet18]

thank you , do you have any suggestions to accelerate my fine tuning ?

[arnfaldur]

Your best bet right now is likely to use a fine-tuning focused tool like unsloth or axolotl. https://github.com/unslothai/unsloth https://github.com/OpenAccess-AI-Collective/axolotl Here is some guide I found on google. https://github.com/modal-labs/llm-finetuning

Doing this with llama.cpp is currently not a good choice. llama.cpp is more focused on inference and accessibility.

[Positronx]

@JohannesGaessler do you project implementing GPU fine tuning soon, or it might take some time?

[JohannesGaessler]

That will depend on a variety of factors, including how much time I will have once I start my PhD. My goal is to have something workable in 2024 and I think I will be able to meet that goal at least.

[Positronx]

Hi @JohannesGaessler , I noticed that the README.md of finetuning specifies that it's possible only for Llama based models. Can I still finetune LLM models based on other architectures (e.g. phi, mistral, gemma)?

[JohannesGaessler]

Don't know, so far I have never touched or read the training code.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.