search

arcee-ai / mergekit

Tools for merging pretrained large language models.
GNU Lesser General Public License v3.0
4.88k stars 446 forks source link

Question about the merge of the Dare method. #373

Open guanfaqian opened 4 months ago

guanfaqian commented 4 months ago

Is there a specific step-by-step procedure or guide for Dare?

It seems to me that what you've written doesn't include the bare reduction parameter and the scaling operation, only the linear and ties merge?