search

yule-BUAA / MergeLM

Codebase for Merging Language Models (ICML 2024)
745 stars 42 forks source link

PEFT integration of DARE method #19

Closed pacman100 closed 7 months ago

pacman100 commented 7 months ago

Hello,

Thank you for the great work with respect to the DARE merging method ✨! Inspired by you, we are planning on integrating DARE merging method for LoRA method in PEFT with PR https://github.com/huggingface/peft/pull/1364. One main usecase of PEFT integration is that it works in an online fashion during runtime instead offline mode of merging state dicts. It would be great if you could review the same and link the integration in your README post once the integration PR is merged.

yule-BUAA commented 7 months ago

Hi,

Thank you for your kind words and appreciation for our work!

We are honored and excited to see that you are planning to integrate it with the LoRA method in PEFT. We would love to review your PR and link it in our README once it is merged.

Please let us know if there is anything we can do to help or support you in this process. We are looking forward to seeing the results of your integration. 😊