lobehub / lobe-chat

🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system. One-click FREE deployment of your private ChatGPT/ Claude application.
https://chat-preview.lobehub.com
Other
44.31k stars 9.93k forks source link

[Request] Set PreferredRegion for OpenAI Chat edge function #470

Closed skyf0cker closed 11 months ago

skyf0cker commented 11 months ago

🥰 需求描述 | Feature Description

Accordding to the Vercel region doc: https://vercel.com/docs/functions/configuring-functions/region, it supports to set the preferred region for edge function so that the function calling will be served by the instance in the specific region. By doing this, we can bypass the OpenAI's block based on region, for example we can set the preferred region to iad1 so that the requests to OpenAI will send from the Washington.

https://github.com/skyf0cker/lobe-chat/blob/515faab30bda5be1974b88e3c94018fedc8d0d75/src/app/api/openai/chat/route.ts#L14-L15

I tested in my self-hosted Lobe-Chat, it looks like that it works. If this really works, maybe we can set it with a default region or provide a method to user to set like through env variables.

🧐 解决方案 | Proposed Solution

we can set the preferred region to iad1 https://github.com/skyf0cker/lobe-chat/blob/515faab30bda5be1974b88e3c94018fedc8d0d75/src/app/api/openai/chat/route.ts#L14-L15

📝 补充信息 | Additional Information

No response

lobehubbot commented 11 months ago

👀 @skyf0cker

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible. Please make sure you have given us as much context as possible.\ 非常感谢您提交 issue。我们会尽快调查此事,并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

arvinxx commented 11 months ago

@skyf0cker yes we can, could your open a pr for this? I think it's great for the block based region user.

arvinxx commented 11 months ago

By the way, I think it's better to use an array regions like ['iad1', 'sfo1'] ?

https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config#preferredregion

skyf0cker commented 11 months ago

@skyf0cker yes we can, could your open a pr for this? I think it's great for the block based region user.

Of course, i will complete this work tonight.

skyf0cker commented 11 months ago

I just want to confirm whether using environment variables to specify the region can be more user-friendly? Some users may not need to consider the limitations brought by regions. If we directly specify the region as the United States, could it potentially lead to longer delays for them when using Lobe-Chat? @arvinxx

arvinxx commented 11 months ago

Some users may not need to consider the limitations brought by regions.

you are right. I think a better way is to add a boolean environment variable to control whether need to use this feature. What's the name of the environment variable should be?

skyf0cker commented 11 months ago

I was thinking of adding an environment variable called "OPENAI_FUNCTION_REGIONS." If this variable is not set, we won't set the preferredRegion, which means we will use Vercel's default logic for accessing the nearest region. If it is set, we will then set the PreferredRegion to the value corresponding to this variable.

lobehubbot commented 11 months ago

✅ @skyf0cker

This issue is closed, If you have any questions, you can comment and reply.\ 此问题已经关闭。如果您有任何问题,可以留言并回复。