Closed farnazj closed 1 month ago
Multimodal is currently not supported on server. The model will generate the response without looking at the image (so it hallucinates)
Related to https://github.com/ggerganov/llama.cpp/issues/8010
Oh no :( what is the latest stable release/commit that still supports the multimodal?
This issue was closed because it has been inactive for 14 days since being marked as stale.
What happened?
When using
./llama-llava-cli
, I get perfectly fine descriptions of images. But when hosting LLava with./llama-server
, LLava hallucinates big time.Here's how I'm running LLava with the cli:
./llama-llava-cli -m models/llava-v1.6-vicuna-7b.Q5_K_S.gguf --mmproj models/mmproj-model-f16.gguf --image images/sth.jpeg -c 4096
Here's how I'm starting the server:
./llama-server -m models/llava-v1.6-vicuna-7b.Q5_K_S.gguf --mmproj models/mmproj-model-f16.gguf -c 2048 --host 127.0.0.1 --port 8000
Here's the python code to send the request:
Name and Version
What operating system are you seeing the problem on?
Mac
Relevant log output
No response