Closed arnavgarg1 closed 4 weeks ago
The current PEFT LoRA modules map to the wrong target modules for Phi-2 since the architecture was recently updated.
I created a PR on the PEFT repo to update this to new target modules: https://github.com/huggingface/peft/pull/1375
For now, we will fall back to target_modules for Phi-2 that lead to good model performance.
The current PEFT LoRA modules map to the wrong target modules for Phi-2 since the architecture was recently updated.
I created a PR on the PEFT repo to update this to new target modules: https://github.com/huggingface/peft/pull/1375
For now, we will fall back to target_modules for Phi-2 that lead to good model performance.