search

arcee-ai / mergekit

Tools for merging pretrained large language models.
GNU Lesser General Public License v3.0
4.15k stars 361 forks source link

Are there any plans to support Encoder-Decoder models like T5? #14

Open Inc0mple opened 7 months ago

Inc0mple commented 7 months ago

Hi, thanks for sharing this awesome work! Are there any plans to support Encoder-Decoder models like T5? Are there any methods for merging such models?

cg123 commented 7 months ago

I do plan to support these kinds of models in the future. I need to hammer out how to adjust the existing architecture to handle them, but it'll happen in the not too far future. (I'm also looking at adding stable diffusion support, as that'll come pretty much free with encoder-decoder models.)

Vasanthengineer4949 commented 6 months ago

Waiting for T5. Hopefully it arrives sooner than expected

shainisan commented 6 months ago

Yes, T5 support would be awesome.

Thanks you for amazing library. Great work!!

init-random commented 1 month ago

+1