search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.4k stars 1.29k forks source link

LoRA on Qwen 2.5 does not patch qkv matrices #1265

Open MinghaoYan opened 2 weeks ago

MinghaoYan commented 2 weeks ago

I am running into the same issue of QKV not getting patched. Any updates on this? Is this due to the bias term in Qwen QKV matrices?

As was pointed out in Issue 1043, Unsloth does not patch QKV matrices.

danielhanchen commented 1 week ago

No need to worry - not a bug! The performance hit is minute - I would say 10% slower. Once all model support comes out, this problem should be fixed