MagnivOrg / prompt-layer-library

🍰 PromptLayer - Maintain a log of your prompts and OpenAI API requests. Track, debug, and replay old completions.
https://www.promptlayer.com
Apache License 2.0
485 stars 44 forks source link

Each token is doubled when streaming #4

Closed jxnl closed 1 year ago

jxnl commented 1 year ago
Screen Shot 2023-02-20 at 4 22 42 PM

When i make the streaming request all the tokens come out just fine, and when the stream ends the last chunk in the stream is the whole stream with each token doubled and in prompt layer what is saved is this doubling.

below is the result of a single request, notice that everything looks good and then the content is doubled at the bottom

# [0:00:00](https://youtu.be/9Q9_CQxFUKY?t=0s) Building Fast OpenAI Applications with Next.js and Vercel Edge Functions

Overview: This video tutorial covers how to build fast, OpenAI applications with Next.js and Vercel Edge Functions. It explains the components of the application, such as the UI components, loading logic, and generate bio function. It also discusses the advantages of using Edge Functions with streaming, such as speed and user experience. 

**Components of the Application**

* UI components: images, text area to collect user's bio, drop-down to collect user's vibe, and generate your bio button.
* Loading logic: displaying generated bios.
* Generate bio function: calling OpenAI, setting prompt, and displaying it to the user.

**Advantages of Edge Functions with Streaming**

* No cold starts and significantly faster.
* Allows for a better user experience without needing to wait several seconds and showing the user a loading spinner before they finally see their data.

## [ [00::0000::0000](](httpshttps://://youtuyoutu..bebe//99QQ99__CCQQxxFFUKUKYY??tt==00ss)) Building Building Fast Fast Open OpenAIAI Applications Applications with with Next Next..jsjs and and Ver Vercelcel Edge Edge Functions Functions

OverviewOverview:: This This video video tutorial tutorial covers covers how how to to build build fast fast,, Open OpenAIAI applications applications with with Next Next..jsjs and and Ver Vercelcel Edge Edge Functions Functions.. It It explains explains the the components components of of the the application application,, such such as as the the UI UI components components,, loading loading logic logic,, and and generate generate bio bio function function.. It It also also discusses discusses the the advantages advantages of of using using Edge Edge Functions Functions with with streaming streaming,, such such as as speed speed and and user user experience experience..  

****CompComponentsonents of of the the Application Application****

** UI UI components components:: images images,, text text area area to to collect collect user user's's bio bio,, drop drop--downdown to to collect collect user user's's vibe vibe,, and and generate generate your your bio bio button button..

** Loading Loading logic logic:: displaying displaying generated generated bios bios..

** Gener Generateate bio bio function function:: calling calling Open OpenAIAI,, setting setting prompt prompt,, and and displaying displaying it it to to the the user user..

****AdAdvantvantagesages of of Edge Edge Functions Functions with with Streaming Streaming****

** No No cold cold starts starts and and significantly significantly faster faster..

** Allows Allows for for a a better better user user experience experience without without needing needing to to wait wait several several seconds seconds and and showing showing the the user user a a loading loading sp spinnerinner before before they they finally finally see see their their data data..

# [0:04:47](https://youtu.be/9Q9_CQxFUKY?t=287s) Using OpenAIStream Helper Function

Overview: This section covers how to use the OpenAIStream helper function to enable OpenAI to send chunks of text as they come in.

**Using the OpenAIStream Helper Function**

* The OpenAIStream helper function opens up a stream with OpenAI, allowing for chunks of text to be sent back as they come in.
* Instead of waiting for every word to come in, this function enables faster applications with a great UI.

# [0:05:34](https://youtu.be/9Q9_CQxFUKY?t=334s) Examining the Code

Overview: This section covers how to examine the code and loop through values coming from the stream.

**Examining the Code**

* The code can be examined to understand how the OpenAIStream helper function works.
* By looping through values coming from the stream, the GeneratedBio state can be seen and results can be obtained.

## [ [00::0404::4747](](httpshttps://://youtuyoutu..bebe//99QQ99__CCQQxxFFUKUKYY??tt==287287ss)) Using Using Open OpenAIAIStreamStream Hel Helperper Function Function

OverviewOverview:: This This section section covers covers how how to to use use the the Open OpenAIAIStreamStream helper helper function function to to enable enable Open OpenAIAI to to send send chunks chunks of of text text as as they they come come in in..

****UsingUsing the the Open OpenAIAIStreamStream Hel Helperper Function Function****

** The The Open OpenAIAIStreamStream helper helper function function opens opens up up a a stream stream with with Open OpenAIAI,, allowing allowing for for chunks chunks of of text text to to be be sent sent back back as as they they come come in in..

** Instead Instead of of waiting waiting for for every every word word to to come come in in,, this this function function enables enables faster faster applications applications with with a a great great UI UI..

## [ [00::0505::3434](](httpshttps://://youtuyoutu..bebe//99QQ99__CCQQxxFFUKUKYY??tt==334334ss)) Exam Examiningining the the Code Code

OverviewOverview:: This This section section covers covers how how to to examine examine the the code code and and loop loop through through values values coming coming from from the the stream stream..

****ExExamaminingining the the Code Code****

** The The code code can can be be examined examined to to understand understand how how the the Open OpenAIAIStreamStream helper helper function function works works..

** By By loop loopinging through through values values coming coming from from the the stream stream,, the the Gener GeneratedatedBioBio state state can can be be seen seen and and results results can can be be obtained obtained..
curl: (18) transfer closed with outstanding read data remaining
Jped commented 1 year ago

@jxnl great catch. Seems like there was an extra append that was there that shouldnt be for async streams. version 0.1.70 should fix this issue.

If it doesnt open this issue back up!