Streaming support for assistants in OpenAI

[code4jhon]

Is there support for stream when setting a component to use an assistant ?

When using the chat (non assistant version) stream works perfectly whether I just set it to true or through simulation but when using an assistant then it doesn't work.

<deep-chat ref="deepChat"
    stream='{"simulation": 10}'
    directConnection='{
        "openAI": {
          "key": "foo",
          "validateKeyProperty": true,
          "assistant": {
            "assistant_id": "bar"
          }
        }
      }'
...

[code4jhon]

I tried this with the latest prod version (1.4.11) and also with the latest dev version (9.0.183)

[OvidijusParsiunas]

Hi @code4jhon. The OpenAI Assistant streaming experience is quite limited in the latest stable version of Deep Chat (deep-chat and deep-chat-react version 1.4.11). We have improved this in the latest dev packages (deep-chat-dev and deep-chat-react-dev) version 9.0.183. The API to use it is slightly different and I can see you almost had it in your example, all you need to do is change the connect property to as follows:

<deep-chat ref="deepChat"
    connect='{"stream": true}'
    directConnection='{
        "openAI": {
          "key": "foo",
          "validateKeyProperty": true,
          "assistant": {
            "assistant_id": "bar"
          }
        }
      }'
...

To note, we will soon be releasing the dev versions to the core version, so you will be able to refer to the documentation. Let me know if this helps your issue.

For more information about Assistant streaming - you can check out the following issue: https://github.com/OvidijusParsiunas/deep-chat/issues/219

[code4jhon]

Thanks for your quick reply @OvidijusParsiunas

Using the connect config stream works nicely with assistant.

Very nice work with the component, thanks!