search

epam / ai-dial-chat

A default UI for AI DIAL
https://chat.lab.epam.com
Apache License 2.0
57 stars 24 forks source link

Historical Assistant's attachments are not passed to core #1307

Open ypldan opened 4 months ago

ypldan commented 4 months ago

EPAM AI DIAL chat version

0.7.2

How to reproduce

  1. Implement application that sends some attachment in response to user message (we've tested with plotly and png attachments).
  2. Reproduce the following dialog:
    • User: some message
    • Assistant: message with attachment
    • User: some message.
  3. In dev console it can be seen that attachment is sent in the request, however if we will check on core/application side - no attachment will be present on assistant's message.

Actual result

  1. Dev console: image
  2. Debug console on application side: image

Expected result

It's expected that assisstant's attachments will be passed to user, since there are business cases when we really need that.

Example: application generates some chart/image that is sent to user, in the next message user asks to explain what is going on on the chart/image and we need to pass it to the image-to-text model (ex. gpt-4-vision).

Confidential information

IlyaBondar commented 2 months ago

DIAL Chat doesn't send assistant attachments as context to API