LoRA+ support - Githubissues

tdrussell / qlora-pipe

A pipeline parallel training script for LLMs.

MIT License

83 stars 8 forks source link

LoRA+ support #20

Closed kallewoof closed 2 months ago

kallewoof commented 3 months ago

Some issues that need addressing, so keeping as draft.

[ ] Deal with model params being passed around and then discarded when LoRA+ is used. Why are these even needed in DeepSpeed space? Can we just None them, or at least None them when LoRA+ is activated?
[x] Wait for merge: https://github.com/huggingface/peft/pull/1973
[ ] Wait for next PEFT release

tdrussell commented 3 months ago

This looks good. Have you tested it yet? One thing to check is that the lora+ optimizer parameters get set correctly. The create_loraplus_optimizer() takes the model instead of the parameters, presumably internally it is still only grabbing the parameters that requires_grad? Need to make sure of that.

kallewoof commented 3 months ago

The create_loraplus_optimizer() takes the model instead of the parameters, presumably internally it is still only grabbing the parameters that requires_grad? Need to make sure of that.

It does that, yes

https://github.com/huggingface/peft/blob/b9260305e3e0129ba35214a40ca7ce60d7ea25b3/src/peft/optimizers/loraplus.py#L70-L83

and it also does bnb module override registration stuff if it is able to (edited to fix link)

https://github.com/huggingface/peft/blob/b9260305e3e0129ba35214a40ca7ce60d7ea25b3/src/peft/optimizers/loraplus.py#L112-L121

kallewoof commented 3 months ago

Have you tested it yet?

I have used this exclusively since I made the pull request, a couple of fine tunes so far, and working on a 70B fine tune right now. The results look fine to me so far, but I will know more after spending some time with it.

kallewoof commented 2 months ago

Training broke down when I tried using this at the recommended ratio 16. Testing other (lower) ratio values to see if they work better. At ratio 5, the training seemed to work, but with lora+ disabled, the training seemed to give slightly better eval loss during the initial stages of training. Will experiment with this some more and do full training runs comparing eval with lora+ disabled and with different ratios.

kallewoof commented 2 months ago

Closing this. Will look into it again when the next PEFT has been released, if I find the time.