search

arcee-ai / mergekit

Tools for merging pretrained large language models.
GNU Lesser General Public License v3.0
4.88k stars 446 forks source link

Support for Vision Model such as ViT #423

Open redagavin opened 2 months ago

redagavin commented 2 months ago

Hi,

Thanks for this wonderful codebase of model merging. I'd like to use it to merge vision models, specifically models that share nearly the same architecture but trained with different objectives, such as CLIP ViT and Dinov2 ViT. I'd like to contribute to the codebase so I plan to write code for this on my own. However, as I read through the codebase, I realize how complex the codebase is. Layers of encapsulation are really a daunting barrier for me to make sensible contribution. Could you please advise me on how to make changes to accommodate my use case?

Thanks a lot!