danny-avila / LibreChat

Enhanced ChatGPT Clone: Features Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. Actively in public development.
https://librechat.ai/
MIT License
19.16k stars 3.19k forks source link

WIP: Local Upload Documents as Input for Models instead of using RAG incas… #4503

Open schnaker85 opened 3 weeks ago

schnaker85 commented 3 weeks ago

Summary

Related Issue: This is the PR for https://github.com/danny-avila/LibreChat/issues/4502 I made it WIP because the discussion in a feature might lead to changes

Changes in the code:

Change Type

Please delete any irrelevant options.

Checklist

Please delete any irrelevant options.

kukjun commented 4 days ago

It seems like a good approach. However, when actually implemented, a problem occurs where the token part becomes NaN when encoding the file.

I think there is a part that processes images as tokens, but there is none for files, so I suggest adding exception handling for this.

// OpenAIClient.js
async buildMessage() {
  ...
  // after 530 line
  // There is a token calculation logic for images, but there is no calculation logic for non-images, so exclude it.
  if (!file.type.startsWith('image')) {
    continue;
  }
  ...
}

Thanks :)