1openwindow / azure-openai-node

This is a fork of the official OpenAI Node.js library that has been adapted to support the Azure OpenAI API. The objective of this library is to minimize the changes required to migrate from the official OpenAI library to Azure OpenAI or revert back to OpenAI.
https://www.npmjs.com/package/azure-openai
MIT License
79 stars 12 forks source link

Streaming tokens received out of order w/ Azure #11

Open joshforman opened 1 year ago

joshforman commented 1 year ago

When talking to my Azure deployment and using streaming mode, I've found that the tokens being returned are out of order. They're all there, just not in the right sequence. Is this a known issue? Here's my code, though it's unremarkable and largely follows the OAI cookbook:

const openaiExplainer = await openai.createChatCompletion({
            model: OPENAI_MODEL_NAME,
            messages: explainerArray,
            temperature: 0,
            stream: true
          }, { responseType: 'stream' });

          openaiExplainer.data.on('data', async (dataExplainer) => {
            const linesExplainer = dataExplainer.toString().split('\n').filter(lineExplainer => lineExplainer.trim() !== '');
            for (const lineExplainer of linesExplainer) {
              const replyExplainer = lineExplainer.replace(/^data: /, '');

              let parsedExplainer;
              try {
                parsedExplainer = JSON.parse(replyExplainer);
              } catch (error) {
                if (replyExplainer != '[DONE]') continue;
              }

              if (replyExplainer === '[DONE]' || (parsedExplainer && parsedExplainer.choices[0].finish_reason)) {
                aiReplyChat.done = true;
                return; // Stream finished
              }

              const deltaExplainer = parsedExplainer.choices[0].delta.content;

              if (deltaExplainer) {
                console.log(deltaExplainer)
                aiReplyChat.explainer += deltaExplainer;
              }
            }
          });

Sample output of the console from the above:

>  Hello
>  ,
>   I
>   Could
>   you
>   add
>   it
>   to
>   my
>   bill
>  ?
>  
>  
>  V
>  ocabulary
>  
>  
>  -
>   mon
>   (
>  m
>  ):
>   my
Fumaz commented 1 year ago

i also have the same issue