search

harveyhuang18 / EMR_Merging

[NeurIPS 2024 Spotlight] EMR-Merging: Tuning-Free High-Performance Model Merging
27 stars 3 forks source link

About number of merged models #2

Open daidaiershidi opened 2 weeks ago

daidaiershidi commented 2 weeks ago

Thank you for the effective work you've brought. I'd like to know if the result of model fusion is multiple adjusted models or a unified model?

harveyhuang18 commented 2 weeks ago

Hello,

Thanks for your attention to our work!

Actually, the reported results are all from the adapted models. The adaptation is quite easy and does not require any additional tuning or training, as follows:

For Task i, we apply the corresponding mask and rescaler to the unified task vector:

task_vector_adpt = vector_unified masks[i] rescalers[I]

In Section 4.5: Ablation Study of our paper, we also report the performance of the unified task vector (not adapted).

Hopefully we have addressed your concern.