🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
The reason, IMO, is that the browser (Firefox 122.0, but surly others as well) doesn't generally allow scripts from one origin ( http://localhost:5173/) to access resources from another origin (the API endpoints) because of the same-origin policy, unless the server supports the CORS protocol and send headers which explicitly allow its resources to be access by other origins.
In the case of simple GET, POST and HEAD request, these headers are sent during the normal client-server dialog. However, the API calls are not simple in this sense, for example because the include an Authorization header. In this cases, the browser crafts a so-called preflight request on its own and sends it to the server to validate the cross-origin use before sending the real request. The preflight request uses the http OPTIONS method.
Here is a picture of the Firefox network log of a working API request towards the openai API endpoint :
And here is one towards the non-functional BricksLLM API endpoint:
The OPTIONS request is replied with a "404 not found" response and the POST request is not even sent by the browser because of its same-origin policy and the failed CORS preflight check.
Please note, that this explanation of same-origin policy and CORS is my current understanding ( source ), but I'm not an expert in that area and may be wrong.
The difference can also be demonstrated with CURL:
I have BetterChatGPT running on http://localhost:5173/. In this ChatGPT-like web application, the AI API calls are sent from the browser. This works when I use https://api.openai.com/v1/chat/completions as the API endpoint. However, it doesn't work, when I use my BricksLLM instance ( https://ai.molgen.mpg.de/api/providers/openai/v1/chat/completions ) as the API endpoint.
The reason, IMO, is that the browser (Firefox 122.0, but surly others as well) doesn't generally allow scripts from one origin ( http://localhost:5173/) to access resources from another origin (the API endpoints) because of the same-origin policy, unless the server supports the CORS protocol and send headers which explicitly allow its resources to be access by other origins.
In the case of simple GET, POST and HEAD request, these headers are sent during the normal client-server dialog. However, the API calls are not simple in this sense, for example because the include an Authorization header. In this cases, the browser crafts a so-called preflight request on its own and sends it to the server to validate the cross-origin use before sending the real request. The preflight request uses the http OPTIONS method.
Here is a picture of the Firefox network log of a working API request towards the openai API endpoint :
And here is one towards the non-functional BricksLLM API endpoint:
The OPTIONS request is replied with a "404 not found" response and the POST request is not even sent by the browser because of its same-origin policy and the failed CORS preflight check.
Please note, that this explanation of same-origin policy and CORS is my current understanding ( source ), but I'm not an expert in that area and may be wrong.
The difference can also be demonstrated with CURL:
Maybe this can be fixed by just adding OPTIONS to the methods which are proxied to the provider.