Merging Lora fine-tuned models with MoE

arcee-ai / mergekit

Tools for merging pretrained large language models.

GNU Lesser General Public License v3.0

4.87k stars 444 forks source link

Merging Lora fine-tuned models with MoE #426

Open AmineBechar07 opened 1 month ago

AmineBechar07 commented 1 month ago

Hello everyone, I have an issue, is there a way to use finetuned models Lora adapters (safetensors) on my own data for merging? Is there someone who managed to realize it?