Bunny v1.1 Llama 3 8b GGUF support/release?

BAAI-DCAI / Bunny

A family of lightweight multimodal models.

Apache License 2.0

921 stars 68 forks source link

Bunny v1.1 Llama 3 8b GGUF support/release? #122

Open ghost opened 2 months ago

ghost commented 2 months ago

Hey all, thanks for your work. It seems there's an official gguf release for the 1.0 version of the Lllama-based model, but not of the 1.1 version. Is that because llama-cpp changes would be required? Or was interest just too low? I'm aware that one can quant one's own models, but lacking experience in this process, an official GGUF release would be quite nice.

Isaachhh commented 2 months ago

The source code of llama.cpp need to be modified to support S2-Wrapper, which is used in Bunny-v1.1.