idea: Allow RAG with .txt files

[4722794]

Problem Statement

Hi, am I missing something?

The current tool only supports pdfs. Why not .txt files?

Feature Idea

Add support for pdfs and .txt files for RAG in tools.

[4722794]

Also, I did go through the documentation here: https://jan.ai/docs/tools/retrieval#enable-the-knowledge-retrieval

But I just don't get it. On my system, there is no provision to select a model. I was hoping to select some embedding model like text-embedding-3.

And where is it storing the chunks? Is it creating a vector database somewhere?

[0xSage]

Apologies, txt rag is not supported yet 🙏

[4722794]

@0xSage no problem; Jan is terrific, thanks for putting it together.

But I've found the RAG implementation in Jan a little difficult to understand;

I've gone through the docs and it's still not clear;

When I upload a file, are you uploading the file to my openai account and using a vector database from there?

Or is it created locally using chromadb?

[imtuyethan]

We should close this ticket & make it a part of https://github.com/janhq/cortex.cpp/issues/1595