search

Azure / enterprise-azureai

Unleash the power of Azure AI to your application developers in a secure & manageable way with Azure API Management and Azure Developer CLI.
MIT License
80 stars 38 forks source link

[Question] Proxy as container app and logging choices in the scenario #67

Closed scdouble closed 2 months ago

scdouble commented 4 months ago

I understand that we are using a container app as a Proxy for load balancing and logging requests between the chat app and AOAI. Could you please share any information about why a Proxy is used as a container app instead of deploying an application gateway? Additionally, why do we choose a container app for logging instead of logging at APIM, considering that APIM's diagnostic logs can be used to aggregate subscription keys for chargeback purposes?

I apologize if I have misunderstood anything. I would greatly appreciate it if you could provide some background on the technology choices made for this scenario.

azureholic commented 4 months ago

Hi @scdouble Thank you for your question. We started working on this repo a while ago. At that time reporting token usage for streaming requests and load balancing in general was not an option, so that's where the proxy comes in which uses YARP. It is still a valid option because the proxy can be used for much more logging scenario's because it "sees" the requests and responses in detail and investigation of that data is being done after the request is send back to the client, so it has a limited impact on the performance.

The API Management team just released GenAI Gateway features that could be an alternative to the proxy solution. Read this blogpost to find out more: https://techcommunity.microsoft.com/t5/azure-integration-services-blog/introducing-genai-gateway-capabilities-in-azure-api-management/ba-p/4146525

Application Gateway (or Azure Frontdoor) would typically be in front of API Management, where the API Management instance is internal. This gives you an extra layer of protection (WAF v2. and mTLS).

Hope this gives you insight on the possibilities.

scdouble commented 4 months ago

@azureholic Thank you very much for your time and assistance ！！Your help has made a positive impact on my understanding and progress.