The streaming response is bugged

[jontstaz]

Hi,

It seems like the raw response from the OpenAI Streaming Response isn't being handled correctly. Refer to my screenshot below:

The raw response from the /api/chat endpoint looks like this:

0:"Richard"
0:" T"
0:"eng"
0:" took"
0:" over"
0:" as"
0:" the"
0:" CEO"
0:" of"
0:" B"
0:"inance"
0:" in"
0:" November"
0:" "
0:"202"
0:"3"
0:","
0:" succeeding"
0:" Chang"
0:"peng"
0:" Zhao"
0:"."
0:" T"
0:"eng"
0:","
0:" who"
0:" h"
0:"ails"
0:" from"
0:" Singapore"
0:","
0:" has"
0:" over"
0:" three"
0:" decades"
0:" of"
0:" financial"
0:" services"
0:" and"
0:" regulatory"
0:" experience"
0:","
0:" having"
0:" held"
0:" key"
0:" positions"
0:" at"
0:" B"
0:"inance"
0:" before"
0:" becoming"
0:" CEO"
0:","
0:" overseeing"
0:" regional"
0:" markets"
0:" beyond"
0:" the"
0:" U"
0:".S"
0:"."
0:" His"
0:" previous"
0:" roles"
0:" include"
0:" Chief"
0:" Regulatory"
0:" Officer"
0:" of"
0:" the"
0:" Singapore"
0:" Exchange"
0:" and"
0:" CEO"
0:" of"
0:" the"
0:" Financial"
0:" Services"
0:" Regulatory"
0:" Authority"
0:" at"
0:" Abu"
0:" Dhabi"
0:" Global"
0:" Market"
0:"."
0:" T"
0:"eng"
0:" is"
0:" focused"
0:" on"
0:" ass"
0:"uring"
0:" B"
0:"inance"
0:"'s"
0:" users"
0:" regarding"
0:" the"
0:" company"
0:"'s"
0:" financial"
0:" strength"
0:" and"
0:" security"
0:"{"
0:"1"
0:"}{"
0:"2"
0:"}."

[BishalSaha]

Try Rewrite and tell me if it persists

[jontstaz]

Try Rewrite and tell me if it persists

Yes it does persist. The OpenAIStream and StreamingTextResponse are seemingly not correctly implemented. I'm not familiar with the library but there seems to be a missing step in regards to processing/parsing/transforming the returned text.

[jontstaz]

Any update?

[BishalSaha]

I did update the code. Changed the formatting basically. If you still face the issue can you please provide steps to reproduce?

Here is my chat: https://omniplex.ai/chat/C50seAJQbs