Files content inside messages?

[mage1028]

Hi, when I use the upload files functions, I can upload all kinds of files. So how can I get these files' content? Thanks very much

[OvidijusParsiunas]

Currently, when transferring files that are not naturally encrypted with base64 or don't have a link reference, it is not possible to observe their internals. I will try to enhance this and update this thread.

[OvidijusParsiunas]

Hi, I have included new functionality inside the deep-chat-dev and deep-chat-react-dev package 9.0.91 version. These packages are exactly the same as the main ones except they just have different names.

The onNewMessage event will now return a file message that contains a new property called ref which is the actual file that was uploaded. New type: {src?: string; name?: string; type?: MessageFileType; ref?: File}.

Let me know if this works for you.

The dev changes should be promoted to the main packages some-time next week.

Thanks!

[OvidijusParsiunas]

This is now available in the main packages version 1.4.5. The MessageFile type documentation has been updated to reflect this.

[rajeshwarpatlolla]

@mage1028 how are able to upload files?

I am able to upload only image files, when i upload pdf or any other files, its giving error.

I am using below code to upload files.

<deep-chat
     class="deep_chat"
     :mixedFiles="true"
     :images="true"
     :directConnection="{
       openAI: {
         key: '<key>',
         chat: { model: 'gpt-4-0125-preview', max_tokens: 2000 },
       },
     }"
></deep-chat>

am i missing something?

[OvidijusParsiunas]

Hi @rajeshwarpatlolla.

I am not quite sure if current GPT-4 models support other file formats other than images (when this comment was posted). For other files, you will need to use an OpenAI Assistant. Make sure to first set up an assistant in the Assistants Playground which accepts files and then refer to it in Deep Chat via assistant_id.

[rajeshwarpatlolla]

I created assitant and gave assistant_id but still same error.

<deep-chat
      class="deep_chat"
      :mixedFiles="true"
      :images="true"
      :assistant="{ assistant_id: 'xxxxxx }"
      :directConnection="{
        openAI: {
          key: 'xxxxxx',
          chat: { model: 'gpt-4-0125-preview', max_tokens: 2000 },
        },
      }"
      :textInput="{ styles: { container: { padding: '10px' } } }"
    ></deep-chat>

Do you have any functioning code samples to help me identify what I might be overlooking, @OvidijusParsiunas, @mage1028 ? Alternatively, if you're aware of any gaps in the provided code, please inform me.

[OvidijusParsiunas]

Hi, you need to use the following configuration:

<deep-chat
  :mixedFiles="true"
  :images="true"
  :directConnection="{
    openAI: {
      key: 'xxxxxx',
      assistant: {
          assistant_id: 'xxxxxxxx',
      },
    },
  }"
></deep-chat>

For the assistant to handle files you will need to create it inside the OpenAI Assistants Playground and reference its id in the assistant_id key. To note, for your assistant to parse files you will need to enable the Retrieval option:

I have tested this locally and it usually works fine with images and csv files. However, pdf files are very inconsistent as it fails to read them most of the time. It is really weird, but that is currently one of the downsides of recent models.

Let me know if this helps. Thanks!

[OvidijusParsiunas]

Another piece of advice I have is: try out the new assistant yourself inside the actual Assistants Playground as the issues with file uploads may be caused by OpenAI API and not Deep Chat itself.

You can also use your browser's Development Console to inspect what the error is.

[rajeshwarpatlolla]

Thank you @OvidijusParsiunas for your response. Now the file upload is working fine, but irrespective of the file I am uploading it is not able to parse or understand the files. Am I missing any additional configuration?

[OvidijusParsiunas]

Hi @rajeshwarpatlolla.

Would you mind testing this out in the Assistants Playground. If uploading files works for you in there, then I can see if there is a potential issue with Deep Chat. But I have personally tried to upload pdfs in the playground lately and have not had much success, hence I think there could be an issue with OpenAI side of things.