enricoros / big-AGI

Generative AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. It features AI personas, AGI functions, multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.
https://big-agi.com
MIT License
5.44k stars 1.24k forks source link

How OpenAI determines the client's IP and geographic location when the server forwards the API request. #595

Open luokelong opened 3 months ago

luokelong commented 3 months ago

We know that OpenAI has disabled API requests in certain regions. What puzzles me is how OpenAI determines the client's IP and geographic location when the server forwards the API request.

[Service Issue] Openai: Forbidden (403): "error": { "code": "unsupported_country_region_territory", "message": "Country, region, or territory not supported", "param": null, "type": "request_forbidden" }
Ig0tU commented 3 months ago

I don't suppose it has something to do with user/email authentication would it? It then pairs emails + other data- geo location included like google does (vpns + google ai = nope.) Then presets regions with api keys when it's applicable-- or-- if just signed in with no use of api key, say with Google, geo-location is absolutely taken into consideration.

Could there be a mechanism in api key generation that maps regions as letter/number combos built in to the keys?

enricoros commented 3 months ago

Since we have no user/pass, I believe OpenAI filters by IP address of the server. If you deploy on localhost, that's gonna be your home IP. If you deploy on Vercel, that's gonna be an "edge server" which is probably in the same region. You could get your own cloud compute machine, install big-AGI on that server, so then independently of where you connect from, your egress to the openai servers is from that region.

luokelong commented 3 months ago

Since we have no user/pass, I believe OpenAI filters by IP address of the server. If you deploy on localhost, that's gonna be your home IP. If you deploy on Vercel, that's gonna be an "edge server" which is probably in the same region. You could get your own cloud compute machine, install big-AGI on that server, so then independently of where you connect from, your egress to the openai servers is from that region.

When I switch VPNs on the client side to different regions and initiate requests, the results are different. So, how does the openai api know that I use VPN to access the transit server (vercel edge server).

enricoros commented 3 months ago

I don't know, however, different regions will connect you to different vercel servers, and OpenAI probably knows those servers.

luokelong commented 3 months ago

I don't know, however, different regions will connect you to different vercel servers, and OpenAI probably knows those servers.

Thank you for your reply.

tranhoangnguyen03 commented 2 months ago

I think the IP for https://get.big-agi.com/ has been banned by Google, Claude, and OpenAI, possibly from too many toxic and erotic requests.

enricoros commented 2 months ago

@tranhoangnguyen03 I've just tried it and it seems to work for me. Do you have any information about this finding?

tranhoangnguyen03 commented 2 months ago

Then i might be mistaken. Sorry for raising false alarms.

I and my friends, all based in Vietnam, have been using get.big-agi.com as our go-to UI for months. Only recently did we begin seeing the "Forbidden" response from Gemini, Claude, and OpenAI. Weird thing is that we have no issue making api calls to all 3 providers directly from the locally (on our laptops) deployed big-agi docker instance.

enricoros commented 2 months ago

Thanks @tranhoangnguyen03 this is great info.

The request from Vietnam goes through Vercel edge servers that are closer: https://vercel.com/docs/edge-network/regions

Do you think it may go through Hong Kong's? I know these have been blocked by OpenAI.

Related? https://vercel.com/changelog/openai-will-not-support-the-hong-kong-region-hkg1-for-functions

enricoros commented 2 months ago

Update: finding how to correctly disallow the hong kong edge functions region here: https://github.com/orgs/vercel/discussions/7451

luokelong commented 1 month ago

Found a temporary solution, export preferredRegion and dynamic in the api/llms/stream/route.ts file, so that the edge function is executed in the specified region. No problems have been found so far.

export const preferredRegion = ['iad1', 'sfo1'];
export const dynamic = 'force-dynamic';