lobehub / lobe-chat

🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system. One-click FREE deployment of your private ChatGPT/ Claude application.
https://chat-preview.lobehub.com
Other
44.04k stars 9.85k forks source link

[Request] 请问有没有办法使要发送的图片不以url形式,而是以base64编码发送? #4305

Closed BiFangKNT closed 3 weeks ago

BiFangKNT commented 3 weeks ago

🥰 需求描述

智谱api不支持ipv6,而我的服务器只支持ipv6访问,因此发送url的话模型是无法读取的,因此希望能够在相关逻辑中加入一个判断,如果是智谱系模型,就先将minio中的图片转为base64编码再发送。或者在智谱的模型设置里添加一个是否将图片url转换为base64的开关。

由于不了解lobe的项目结构,我不知道相关的处理逻辑是在哪里的,希望大佬告知一二!

相关讨论:https://github.com/lobehub/lobe-chat/issues/3909#issuecomment-2381260111

🧐 解决方案

在相关逻辑中加入一个判断,如果是智谱系模型,就先将minio中的图片转为base64编码再发送。或者在智谱的模型设置里添加一个是否将图片url转换为base64的开关。

📝 补充信息

No response

lobehubbot commented 3 weeks ago

👀 @BiFangKNT

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible. Please make sure you have given us as much context as possible.\ 非常感谢您提交 issue。我们会尽快调查此事,并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

lobehubbot commented 3 weeks ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


🥰 Description of requirements

The Wisdom Spectrum API does not support IPv6, and my server only supports IPv6 access. Therefore, the model cannot be read if the URL is sent. Therefore, I hope to add a judgment to the relevant logic. If it is a Wisdom Spectrum model, first change the Minio Convert the image to base64 encoding before sending. Or add a switch in the model settings of Zhipu whether to convert the image URL to base64.

Since I don't understand the project structure of lobe, I don't know where the relevant processing logic is. I hope you can tell me something!

Related discussion: https://github.com/lobehub/lobe-chat/issues/3909#issuecomment-2381260111

🧐 Solution

Add a judgment to the relevant logic. If it is an intellectual genealogy model, convert the image in the minio to base64 encoding before sending it. Or add a switch in the model settings of Zhipu whether to convert the image URL to base64.

📝 Supplementary information

No response

arvinxx commented 3 weeks ago

不考虑做,等智谱支持 ipv6 就行了。或者你换成ipv4也行。

另外现在已经有环境变量支持url转base64了

lobehubbot commented 3 weeks ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Don't think about it, just wait until Zhipu supports ipv6. Or you can switch to ipv4.

In addition, there are now environment variables that support url conversion to base64.

BiFangKNT commented 3 weeks ago

不考虑做,等智谱支持 ipv6 就行了。或者你换成ipv4也行。

另外现在已经有环境变量支持url转base64了

@arvinxx 请问这个环境变量是在哪里添加的?是全局的吗?之前好像看到过相关的讨论,但是找不到了

lobehubbot commented 3 weeks ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Don’t consider doing it, just wait until Zhipu supports ipv6. Or you can switch to ipv4.

In addition, there are now environment variables that support url conversion to base64.

Where is this environment variable added? Is it global? I seem to have seen related discussions before, but I can’t find them.

BiFangKNT commented 3 weeks ago

@arvinxx 请问是LLM_VISION_IMAGE_USE_BASE64这个环境变量吗?这个好像是全局的,我不太想让所有模型都转base64

lobehubbot commented 3 weeks ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@arvinxx Is it the environment variable LLM_VISION_IMAGE_USE_BASE64? This seems to be global. I don’t want all models to be converted to base64.

BiFangKNT commented 3 weeks ago

@arvinxx 另外大佬,我开启了这个环境变量,也还是不行 image image

lobehubbot commented 3 weeks ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@arvinxx Another guy, I enabled this environment variable, but it still doesn’t work. image image

BiFangKNT commented 3 weeks ago

@arvinxx 诶,关闭客户端请求模式就可以了,这是为什么啊? image image

lobehubbot commented 3 weeks ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


@arvinxx Hey, just turn off the client request mode. Why is this? image image

arvinxx commented 3 weeks ago

https://github.com/lobehub/lobe-chat/discussions/3888

lobehubbot commented 3 weeks ago

✅ @BiFangKNT

This issue is closed, If you have any questions, you can comment and reply.\ 此问题已经关闭。如果您有任何问题,可以留言并回复。

BiFangKNT commented 3 weeks ago

3888

@arvinxx 好的,非常感谢!

lobehubbot commented 3 weeks ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


3888

@arvinxx Ok, thank you very much!