search

OvidijusParsiunas / deep-chat

Fully customizable AI chatbot component for your website
https://deepchat.dev
MIT License
1.26k stars 170 forks source link

Assistants Multiple Response Loading Bug #159

Open jackitaliano opened 2 months ago

jackitaliano commented 2 months ago

Scenario

Recreate

there are other ways to recreate, this one seems most consistent

can also make this happen by uploading your own csv and then asking for the same

Thoughts

It seems that the run is complete when the message is returned, so I'm guessing it's not an issue of not waiting for the run to finish.

In that case, my guess is that multiple text responses may not be handled. I.e. deep-chat has the message, but only adds the most recent message from the thread instead of all the recents from the latest run. Could very well be wrong.

Not incredibly urgent, but hoping for a fix.

Thanks!

OvidijusParsiunas commented 2 months ago

Deep Chat only looks at the last message from that thread. The fix for this should be quite simple, but I will exercise caution. This also does not appear to work for streams, though the solution for that might be much more complex. I will update you on the progress. Thanks!

jackitaliano commented 2 months ago

Deep Chat only looks at the last message from that thread. The fix for this should be quite simple, but I will exercise caution. This also does not appear to work for streams, though the solution for that might be much more complex. I will update you on the progress. Thanks!

That's kinda what I suspected. Again, no rush, especially with the new job you mentioned in other issue.

Thanks!

OvidijusParsiunas commented 2 months ago

Hi, I have managed to fix this for regular requests in deep-chat-dev and deep-chat-react-dev version 9.0.159. Please let me know if you see any issues. I will investigate the solution for streaming tomorrow. Thanks!

OvidijusParsiunas commented 2 months ago

Hi, I have now updated the deep-chat-dev and deep-chat-react-dev packages to version 9.0.160 to allow streaming with files. Let me know if this works for you. Thanks!

jackitaliano commented 2 months ago

After testing this out, there seems to be an issue that this fix creates.

When the assistant returns an image file, it returns multiple of the same image. I've tried recreating this on prod, and this issue does not appear.

Recreate

  1. Send message "Give example data for a csv and write it to a file. Then make a suitable bar chart for the example data."
  2. Assistant returns response with text, one csv, and two of the same images.
    • can put a console.log(response) in response interceptor, which shows that the response has a duplicate image.
    • this can also be seen with a console.log(message) in onMessage or onNewMessage (depending on version)

This appears in deep-chat-dev v9.0.160, but does not appear in deep-chat v1.4.11.

OvidijusParsiunas commented 2 months ago

Hey @jackitaliano.

I can see the issue. I'm a little out of time today but I'll investigate it as soon as I get a chance tomorrow. Thanks!

jackitaliano commented 2 months ago

Hello @OvidijusParsiunas ,

Sounds good, thanks for looking into!

OvidijusParsiunas commented 2 months ago

Hi @jackitaliano.

The issue should be fixed in version 9.0.165. Let me know if this works for you and if you encounter any issues. Thanks!

jackitaliano commented 2 months ago

Hello @OvidijusParsiunas,

This issue appears to still be present.

Screenshot 2024-04-22 at 4 16 10 PM

In this chat, there are only the two files (image and csv), but it loads the image twice.

Hope this isn't too much of a hastle.

Thanks!

Edit: To be clearer, this issue doesn't seem to be an edge case. If you make the same request as the user, it'll usually respond like that. Also, the same occurs if there are multiple text responses.

OvidijusParsiunas commented 2 months ago

Hi @jackitaliano.

Thankyou for letting me know about this issue. Deep Chat has been configured to parse the incoming response message and if it identifies a reference to a file it automatically creates another message with that file to allow the user to download. However it appears that OpenAI assistants are automatically sending another message with the file contents for images. I was able to fix this problem and have released it in version 9.0.167. I have also noticed that when there are multiple response messages, the messages (except the last one) are displayed in the wrong order. This has also been fixed.

Through testing I have also found that when streaming, the response can sometime stop and not generate other messages e.g. the bar chart message. It appears this happens when there is a "server error" response. Hence, unfortunately this is on the OpenAI side of things, so if the stream does not return the full result - Deep Chat can't do much about it.

Thankyou for your feedback and let me know if you notice any other issues in the future.