Model with character cards could not read images.

[Naozumi520]

As title, I loaded a model (Yi 34b) with mmproj, and it's working surprisingly good. However, I encountered an issue when attempting to load the model with a custom character card. The model was unable to accurately identify the elements present in the image, but without character card, everything's good so far.

[LostRuins]

Did you enable "AI vision" to "LLaVA" for the image inside the card you want to use?

Also, kindly send a link to the mmproj file you used. As I would like to catalogue all of them.

[Naozumi520]

Yes, I'm sure that "AI vision" is enabled by selecting "LLaVA". Not only unable to accurately identify the elements, but it will also saying something doesn't exist.

Also, here is the mmproj (mmproj-model-f16.gguf)

Thank you for your hard work!

[LostRuins]

Hmm, and this image is added as a regular image?

I can think of some possibilities:

• Image is not an inserted image, but the character portrait. Only inserted images will be sent to the backend. If you want to submit the character portrait, screenshot or save it, and then upload that file instead.
• AI vision mmproj not correctly loaded. Make sure it says "Active" when you select LLaVA on that image, not "unsupported"
• Too many images: KoboldCpp only allows submitting max 4 images per context currently. If you have more than that, try disabling or removing some earlier images. They will be submitted in sequence.
• Too small context: LLaVA 1.6 is very expensive and cost about 2000 tokens per image. So if your context size is too small then it will not be able to fit your images. To solve that, submit fewer images or increase the max context size.