search

arcee-ai / mergekit

Tools for merging pretrained large language models.
GNU Lesser General Public License v3.0
4.87k stars 444 forks source link

Request add support for Variation Ratio Merge (VARM) method #262

Open tony92151 opened 7 months ago

tony92151 commented 7 months ago

Thanks for this amazing work. It makes everything easier to merge models.

I read this paper recently and the proposed method, Variation Ratio Merge (VARM), is also a novel merge method. https://arxiv.org/abs/2402.16107 it will be helpful if mergekit can support VARM.

Thank

cg123 commented 7 months ago

Thanks for pointing this out, I'll look into adding it soon!