MagnivOrg / prompt-layer-library

🍰 PromptLayer - Maintain a log of your prompts and OpenAI API requests. Track, debug, and replay old completions.
https://www.promptlayer.com
Apache License 2.0
503 stars 46 forks source link

Async Streaming yields the complete response after streaming it. #5

Closed jxnl closed 1 year ago

jxnl commented 1 year ago

If a stream is supposed to generate "A B C D" as tokens, i expect the yields to be

A 
B
C
D

instead i get

A 
B
C
D
A\n\B\nD\C

the last yield is the whole result which makes the application double the result.

Jped commented 1 year ago

@jxnl good catch, we were doing a shallow instead of deep copy. Should be fixed with version 0.1.71 please open back up if it doesn't.