How does 'Merging in the Parameter Space' method merge parameters from two models?

SakanaAI / evolutionary-model-merge

Official repository of Evolutionary Optimization of Model Merging Recipes

Apache License 2.0

1.21k stars 88 forks source link

How does 'Merging in the Parameter Space' method merge parameters from two models? #6

Open PPMark0712 opened 5 months ago

PPMark0712 commented 5 months ago

Thank you for providing innovative ideas. Is there any code available for merging models? I couldn't seem to find it in this GitHub repository. I am very interested in your method of using evolutionary algorithms to merge models, especially curious about how the 'Merging in the Parameter Space' method merging the weights from two models. Could you please provide further explanation?

wangyuchen333 commented 5 months ago

https://huggingface.co/blog/mlabonne/merge-models this might help