Add qlora to our current codebase

[samsja]

Context

we want to add qlora (lora + 4 bits int quant) to our codebase.

The goal is to reduce memory usage and the cost of finetuning without degrading quality.

paper blog post

Steps

• [x] activate 4bits in Peft. Should be as easy as turning on a flag
• [x] run the modal on one GPU for one epoch and look at memory consumption compared to 8 bits training
• [x] run a full training (3 epochs) with the same parameters as 8bits training and compare result to see if we don't degrade quality

[alaeddine-13]

you can get inspired by this PR: https://github.com/tloen/alpaca-lora/pull/487