Could you please share any specific guidelines on doing this with PEFT/ LoRA? Particularly the LoRA config e.g.target_modules=["q_proj","v_proj"] or something else. I notices the finetuning script provided and is much appreciated but some example on how to do this with more resource efficient methods such as LoRA would be awesome. Thanks in advance!
Could you please share any specific guidelines on doing this with PEFT/ LoRA? Particularly the LoRA config e.g.target_modules=["q_proj","v_proj"] or something else. I notices the finetuning script provided and is much appreciated but some example on how to do this with more resource efficient methods such as LoRA would be awesome. Thanks in advance!