search

yule-BUAA / MergeLM

Codebase for Merging Language Models (ICML 2024)
745 stars 42 forks source link

Can models of different architectures be merged? #7

Closed win10ogod closed 9 months ago

win10ogod commented 9 months ago

Can models of different architectures be merged?

yule-BUAA commented 9 months ago

Hi. Models with different architectures cannot be merged by existing model merging methods since they assume the models to be merged are fine-tuned from the same architecture. They merge parameters of multiple models so they need to guarantee the architecture of the models to be identical.

win10ogod commented 9 months ago

thank!