Is there any plan to replace the language model with a smaller one?

baaivision / Emu

Emu Series: Generative Multimodal Models from BAAI

https://baaivision.github.io/emu2/

Apache License 2.0

1.66k stars 86 forks source link

Is there any plan to replace the language model with a smaller one? #53

Closed xmy0916 closed 11 months ago

xmy0916 commented 11 months ago

eg. Yi-6B, Yi-6B-chat, Mistral-7B.

Quan-Sun commented 11 months ago

We don't have plans for a 6B version. We're currently training a smaller version of Emu2 using Llama2-13B, and we plan to release it once the training and evaluation processes are complete.