search

Azure / enterprise-azureai

Unleash the power of Azure AI to your application developers in a secure & manageable way with Azure API Management and Azure Developer CLI.
MIT License
80 stars 38 forks source link

[Question] Streaming support #59

Closed codylittle closed 6 months ago

codylittle commented 6 months ago

Under the Key Features, it outlines that streaming requests are supported for all features.

However, after reading the APIM policy, it reads as if streaming requests are blocked.

I'm yet to deploy this, so forgive me if this is just "APIM Magic". But, are streaming completions actually supported?

azureholic commented 6 months ago

Hi @codylittle It is supported. The policy you mention is not used in the deployment (nice find though!) The policy that is used is the api-policy-chargeback.xml that uses the proxy. That proxy supports streaming and token calculation for streaming.

codylittle commented 6 months ago

Thanks @azureholic for the quick response - looking forward to setting this up