lobehub / lobe-chat

🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system. One-click FREE deployment of your private ChatGPT/ Claude application.
https://chat-preview.lobehub.com
Other
44.37k stars 9.94k forks source link

[Bug] Models from OpenRouter that support image input not being identified as having image capability in LobeChat. #2405

Closed trilin6 closed 6 months ago

trilin6 commented 6 months ago

💻 Operating System

Windows

📦 Environment

Vercel / Zeabur / Sealos

🌐 Browser

Chrome

🐛 Bug Description

As mentioned in my discussion post, i think this may be a bug so submitting this issue in hope someone could confirm if this is intended functionality or how to fix it.

Discussed in https://github.com/lobehub/lobe-chat/discussions/2404

Originally posted by **trilin6** May 6, 2024 # Vision models not identifying as supporting images, disabling image input functionality -> Recently started using LobeChat, and have ran into an issue. Connecting to openrouter, I see some models like GPT-4 Vision correctly identify as accepting images in the model list, however Claude 3 Opus which clearly supports image input in openrouter has this disabled in LobeChat. I am unable to find any way to override this manually and am not sure if this is intentional for some reason or a bug. Could anyone inform me whether this is something I'm able to fix? Thank you. ![2024-05-06 20 59 56 - firefox](https://github.com/lobehub/lobe-chat/assets/27000192/5bc0a5c9-ce88-46e2-b989-e2918a22d4a5) ![2024-05-06 21 02 43 - firefox](https://github.com/lobehub/lobe-chat/assets/27000192/189c7966-b3ce-4cf3-a02e-fd683f319da5)

🚦 Expected Behavior

If a model supports image input, green eye logo should appear by model name in dropdown and image functionality should be enabled.

This is the case for GPT-4 Vision model from OpenRouter, but Claude 3 which also supports images does not display as such.

📷 Recurrence Steps

No response

📝 Additional Information

No response

lobehubbot commented 6 months ago

👀 @trilin6

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible. Please make sure you have given us as much context as possible.\ 非常感谢您提交 issue。我们会尽快调查此事,并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

arvinxx commented 6 months ago

currently we use text match to check whether the modal have vision. I can fix it

lobehubbot commented 6 months ago

✅ @trilin6

This issue is closed, If you have any questions, you can comment and reply.\ 此问题已经关闭。如果您有任何问题,可以留言并回复。

lobehubbot commented 6 months ago

:tada: This issue has been resolved in version 0.155.6 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

sazonovanton commented 2 months ago

Issue is there (v1.12.14)

image