search

enricoros / big-AGI

Generative AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. It features AI personas, AGI functions, multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.
https://big-agi.com
MIT License
5.22k stars 1.18k forks source link

Expand data sources for personas #372

Open arbiterFF opened 7 months ago

arbiterFF commented 7 months ago

Why Prompt is a rather shallow definition of persona. I'd like to be able to connect data sources (e.g. PDFs, google docs and sheets) to make experts that I can then potentially even sell. The value is in the data.

Description Introduce new data sources in the persona definition:

Ideally in the long run we can also include user preferences / memory + frameworks.

enricoros commented 7 months ago

This depends on having RAG systems. The next step in Big-AGI is to have a HAG system via a workspace implementation, where the user can curate highly meaningful snippets of text (some call then "knowledge files"). The architecture for the latter is almost defined, but would take a few weeks for the implementation. HAG is phase 1.

Other integrations will probably require a backend side complete with Storage (e.g. Amazon S3/Uploadthing), Work queues, OAuth authentication for intergations, integration code, the RAG system itself (currently many implementations, but they all fall short, especially in "scoping" RAG to a Persona or Chat or Project).

Adding to the roadmap, but this requires a full rewrite of Big-AGI.

cachatj commented 2 months ago

very interested in pointing Big-AGI to documentation (confluence, markdown), git repos (github, gitlab), PDF libraries (directory) & more. Could really build out this functionality with LlamaIndex fairly easily.

@enricoros - How far off is the HAG System / workspace implementation described above? Looks like last update was Feb 2, 2024.....

enricoros commented 2 months ago

@cachatj depends on what you need Big-AGI to do. Right now doing RAG ourselves is not in our scope.

You may be able to diy with standing up an openai compatible endpoint and chatting with that model.