[FR] Localizing Communication with API Endpoint in Virtual Network Integration

[suzuki-shm]

This issue may be related to #722, #720, #708, #644, #635.

Hello, I am currently utilizing Azure OpenAI Service behind API Management for logging and analytics purposes. I have adjusted the query parameters to be equivalent to those of Azure OpenAI Service, enabling execution through the following curl command:

curl -X POST "https://MY_API_MANAGEMENT_DOMAIN.azure-api.net/openai/deployments/gpt-35-turbo/chat/completions?api-version=2023-05-15" \
     -H "Content-Type: application/json" \
     -H "api-key: MY_API_MANGEMENT_SUBSCRIPTION_KEY" \
     -d '{"messages":[{"role":"system","content":"string","name":"string"}]}'

However, I encounter an 'Invalid credentials' error when trying to use Azure OpenAI Service through Cursor. My input settings are as follows:

• Base URL: https://MY_API_MANAGEMENT_DOMAIN.azure-api.net/
• Deployment Name: gpt-35-turbo
• API Key: MY_API_MANGEMENT_SUBSCRIPTION_KEY

I have confirmed that authentication works for Azure OpenAI Service behind the API Management.

How does Cursor authenticate with the Azure OpenAI Service? Understanding this process might help me to implement it through the API Management Service as well. One point of concern for me is the specification of the api-version. If authentication is tied to a specific version, it would be very helpful if you could share that information!

---

P.S.1: I suspected that an HTTP request was being sent to API Management with each attempt to toggle it 'On', so I checked this using Log Analytics and found that they are indeed successfully returning a 200 status for OPTIONS method. P.S.2: I have noticed Cursor uses 2023-03-15-preview for Azure OpenAI Service, but it still cannot auth even if I have added the version to APIM. P.S.3: I studied CORS and now I doubt that the CORS setting in APIM does not match with Cursor. What the domain is used to contact with Azure OpenAI Service?

[suzuki-shm]

I came up with the idea of using the Developer Tools and, upon checking the network status, I discovered that vscode-file://vscode-app serves as the Origin. Consequently, I added this to the CORS settings in API Management, which successfully enabled authentication for API Management as well.

However, I am still unable to perform question and answer sessions using Cursor. When I attempt to execute it, the following error appears:

Upon checking with Developer Tools again, I encountered the following error:

ConnectError: [unavailable] getaddrinfo ENOTFOUND MY_API_MANAGEMENT_DOMAIN
    at p (workbench.desktop.main.js:4:106346)
    at c (workbench.desktop.main.js:4:114933)
    at Object.read (workbench.desktop.main.js:4:113224)
    at async Object.next (workbench.desktop.main.js:5:940)
    at async c.w (workbench.desktop.main.js:21959:14023)
    at async Se.streamResponse (workbench.desktop.main.js:21247:5089)
    at async workbench.desktop.main.js:21348:10838
    at async Se.streamLines (workbench.desktop.main.js:21294:9758)
    at async O.streamDiff (workbench.desktop.main.js:21336:11571)
    at async workbench.desktop.main.js:21349:163
    at async Ue (workbench.desktop.main.js:22365:8806)

It seems to be a address resolution issue. Indeed, this API Management is enclosed within our company's intranet and is not accessible from the internet. However, this domain is registered with our company's DNS server and is also listed in the hosts file of my Windows machine.

Why would this error occur? Any feedback or suggestions are highly welcome. Thank you!

[jakobFNF]

Cursor sends the requests from its own backend, even if you use your own API key. That might be the problem.

[suzuki-shm]

@jakobFNF Thanks.

I am surprised to discover that queries are being sent from Cursor's backend.

My API Management Service and Azure OpenAI Service are integrated into a virtual network, rendering them inaccessible from the internet. This might be the reason why I am unable to access them as you say. For enterprise applications, such a setup is quite common, so I am hoping for a solution that localizes communication with the API endpoint as well!

[itxch]

+1. Is there any way around this? We have Azure Open AI services that are only accessible via private link?