CORS Policy preventing LLM queries to Azure OpenAI

[zoeqevans]

I was trying to run the example queries on the /play page, and ran into CORS issues with all my requests. I've pasted the chrome dev console logs below, but this is broadly what I expect: CORS should prevent the page from making a javascript request to an OpenAI endpoint, right?

Tools like these usually work around CORS by proxying the requests via a non-CORS-violating backend, but it looks like that's not happening here. Am I misunderstanding CORS / doing something strange here. How could this ever have worked?

Querying Azure OpenAI deployed model 'gpt-35-turbo' at endpoint 'https://api.llm.palantir.tech/preview' with prompt 'What is the opening sentence of The Poppy War by R.F.Kuang?'...

Access to fetch at 'https://api.llm.palantir.tech/preview/openai/deployments/gpt-35-turbo/chat/completions?api-version=2023-03-15-preview' from origin 'https://chainforge.ai' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

[ianarawjo]

Hi Monty --Yes, you are correct that some providers block CORS requests. However, OpenAI, HuggingFace, and Google PaLM2 all allow it. We route Anthropic through the server, because Anthropic has a CORS blocker.

In the past, others have used Azure OpenAI endpoints on the locally installed version of ChainForge without incident. However, I'm not aware of anyone trying it through the /play page.

Based on this context, it could be one of three things:

• the particular server you are querying blocks CORS requests, but other Azure OpenAI servers do not (this is possible, as it seems you have turn on CORS for specific endpoints https://learn.microsoft.com/en-us/answers/questions/1329470/cant-turn-on-cors-from-adding-data-source-dialog-i ; https://learn.microsoft.com/en-us/azure/container-apps/cors?tabs=arm&pivots=azure-portal)
• Microsoft has recently changed the CORS headers for Azure OpenAI and now prevents such requests for all users
• for some reason, the locally installed version of ChainForge works fine with CORS for Azure OpenAI endpoints, but the /play one does not

Unfortunately, this is more than a simple fix. Routing requests would involve removing a dependency on Azure OpenAI's TypeScript API, and transitioning to bare REST calls instead. I'm on vacation at the moment, and won't be able to handle this until next week.

In the meantime, try installing CF locally and/or investigating whether CORS policy can be enabled on your Azure endpoint. If it still doesn't work, let us know here.

NOTE: I personally don't have access to Azure OpenAI, which makes testing its REST API difficult (it's been a longstanding issue that I raised w/ the Azure team --Microsoft is not willing to give us a dummy endpoint, so I'm at wit's end).