Issue: When using image , the response is missing.

dusty-nv / NanoLLM

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

https://dusty-nv.github.io/NanoLLM/

MIT License

168 stars 24 forks source link

Issue: When using image , the response is missing. #36

Open vseranvz opened 3 weeks ago

vseranvz commented 3 weeks ago

Hi, I am using nanollm container on Jetpack 6 rev 1 . By itself it is functioning perfectly fine.

But when I tried to use my example/unittest python file inside the docker container, text prompts are good. But images don't give out any response. I tried both LLAVA and VILA.

Any help is appreciated.

Thank You, Vee

dusty-nv commented 3 weeks ago

Hi @vseranvz, the image prompt needs to be followed by a text prompt (the question about the image). Or does it reply, but not about the image?