search

Azure / apim-landing-zone-accelerator

The Azure API Management landing zone accelerator provides an architectural approach and reference implementation to prepare landing zone subscriptions for a secure API Management infrastructure.
https://aka.ms/apim-lza
MIT License
148 stars 153 forks source link

Support for openai streaming responses in genai scenarios #128

Closed kshitijcode closed 2 months ago

kshitijcode commented 4 months ago

Currently we get an internal server error

Expression evaluation failed. The message body is not a valid JSON. Unexpected character encountered while parsing value: d. Path '', line 0, position 0. at Newtonsoft.Json.JsonTextReader.ParseValue

on streaming use cases in https://github.com/Azure/apim-landing-zone-accelerator/blob/main/scenarios/workload-genai/policies/genai-policy.xml

`

@{ JObject api_response = context.Response.Body.As<JObject>(preserveContent: true); return api_response.ToString(); } ` We need to put a condition that this only applies for non streaming scenarios.
prasann commented 2 months ago

discussed with @kshitijcode , not picking up this issue for now.