Support for LLAMA V.1.5 Multimodal LLMs

ggerganov / llama.cpp

LLM inference in C/C++

MIT License

67.23k stars 9.65k forks source link

Support for LLAMA V.1.5 Multimodal LLMs #3778

Closed Dyl777 closed 7 months ago

Dyl777 commented 1 year ago

Curious if MLLMs can work on it. I am already supposing LLAMA V1.5 can't . I can suggest checking out more efficient MLLM models like X-LLM

Green-Sky commented 1 year ago

are you talking about LLAVA ? because basic support has been merged. see https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/README.md

Dyl777 commented 1 year ago

Thks for pointing that out but not just for LLAMA V1.5, am referring to a format for MLLMs in general being released

johnwick123f commented 11 months ago

@Dyl777 there is no llama v1.5 format? I guess you might be talking about llava 1.5 or vicuna 1.5 which many mllms use as there llm. however so far any mllm that is similar to llava like sharegpt4v or obsidian should work currently. However these models work the same way but just use different clip encoders or different llms.

Currently tho, any other architecture wont work but i think support should come soon.

github-actions[bot] commented 7 months ago

This issue was closed because it has been inactive for 14 days since being marked as stale.