search

lmstudio-ai / lmstudio-bug-tracker

Bug tracking for the LM Studio desktop application
10 stars 3 forks source link

Running vision models is a problem? #101

Open cardchase opened 2 months ago

cardchase commented 2 months ago

Failed to load model: No LM Runtime found for format 'safetensors!

Model: Phi-3.5-vision-instruct-gguf

image

image

Model downloaded from the "Discover" tab

yagil commented 2 months ago

@cardchase does any other model load successfully?

cardchase commented 2 months ago

@cardchase does any other model load successfully?

Yes. the rest of the models i.e.

image

loads successfully.

YorkieDev commented 2 months ago

Hi @cardchase None of the Phi 3.5 Vision or MoE models will load in LM Studio. Support for the model isn't implemented in llama.cpp yet. https://github.com/ggerganov/llama.cpp/issues/9119

Longhuiberkeley commented 1 month ago

Joining this convo a bit late. Is there any vision model that works and would recommend?

YorkieDev commented 1 month ago

@Longhuiberkeley these work well within LM Studio: https://huggingface.co/collections/lmstudio-ai/vision-models-gguf-6577e1ce821f439498ced0c1

Mini CPM 2.6 is another good model (one of the better local vision models) That works: https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf

Longhuiberkeley commented 1 month ago

@YorkieDev That's a speedy reply!! Thank you so much. I am going to download them and test them out now! 👍