Feature Request: Enable upload of files to chat with

[fsagini]

Would be great to have the user click on an upload button next to the input box and upload a file, which then gets embedded and can be used as context in the chat

[VatsaDev]

Thats a great feature, use it alot in claude. But isn't it not nearly as useful as it is with claude, because of How claude has a 100K context length, and it's good at document QA, while Llama has no training on that, and has a context length of 4096, which will fill up quick with documents.

[julien-blanchon]

This PR is a draft for this feature: https://github.com/huggingface/chat-ui/pull/449, you can find a gif video in the corresponding issue https://github.com/huggingface/chat-ui/issues/441.

This is even going bit further than the Claude implementation as this PR leverage Mathpix to convert the pdf to a markdown file with the latex equation and table in it. Claude just extract pure text from the document and don't process equation and table. However Mathpix is a paid API and most user don't need such powerfull transform.

If many peaple are looking for this feature might update the PR or open a new one with a more simple implementation, more like the Claude one.

If you want you can also improve my PR

[oxaronick]

It might be worth looking at the open-webui feature. It allows you to upload a PDF to a conversation and does RAG using the document for any messages in that conversation.

I would prefer this to having a PDF converted to markdown within the chat. My use case is bascially uploading PDFs of several hundred pages and having the LLM act as a librarian for that data.

Alternatively, if chat-ui could call an external URL to get the augmented prompt, I'd also be happy with that. However, I don't know if there is any sort of standard API for prompt augmentation. Seems like it's included within most apps like chat-ui.

[shrwnsan]

Just saw this earlier, "upload file" button in chat

🎉