Setup the apps for George AI

[moncapitaine]

• [ ] GraphQL API
• [ ] UI for managing the Documents, upload, delete, download of PDF files plus managing the system prompt.
• [ ] UI for chatting about the documents
• [ ] UI for generation of target formats like 'Travel Plan'

Publishing company will upload and maintain PDF documents Publishing company will develop UI's based on our example UIs like the chatbot and the summary generator

Implementation ideas:

• typesense as a a document/vector store
• SQL lite a s a document/vector store
• ....

[PascalHelbig]

Here is my idea:

flowchart TD
  subgraph pocketbaseGroup[Pocketbase]
    pocketbase[Pocketbase 📦]
    pocketbaseDb[(Pocketbase Database 🗄️)]
    pocketbase --> pocketbaseDb
  end

  pocketbase -- PDF uploaded/updated/deleted--> graphql
  pdfProcessor -- PDF parsed & processed--> pocketbase

  subgraph backend[Backend]
    subgraph llmService[LLM Service 🛠️]
      graphql[GraphQL Endpoint 🌐]
      pdfProcessor[PDF Processor 📄]
      chains[Chains 🔗]

      graphql --> pdfProcessor
      graphql --> chains
    end

    llmDb[(LLM Database 🗃️)]
    pdfProcessor -- write docs with embeddings --> llmDb
    chains -- use db as retriever --> llmDb
  end

  subgraph frontend[Frontend 💻]
    chatbot[Chatbot 🤖]
    travelPlanner[Travel Planner 🗺️]
  end

  chatbot --> graphql
  travelPlanner --> graphql

Components

• Pocketbase 📦
  • used by the publisher
  • used for uploading PDFs
  • stores PDFs locally
  • it will inform the LLM Service about the uploaded PDFs
• Pocketbase Database 🗄️
  • stores Pocketbase data using sqlite
• LLM Service 🛠️
  • on backend service
  • consists of three components: GraphQL Endpoint, PDF Processor, Chains
• GraphQL Endpoint 🌐
  • communication endpoint of the LLM Service
• PDF Processor 📄
  • processes the uploaded PDFs
  • extracts the text and embeddings
  • writes the extracted data and the embedding to the LLM Database
  • informs Pocketbase that the PDF has been processed
• Chains 🔗
  • uses the embeddings in LLM Database as a retriever
  • contains the chains for chatbot and travel planner
• LLM Database 🗃️
  • stores the extracted data and embeddings
  • must be database with vector search support
• Frontend 💻
  • one Frontend App with two routes: Chatbot and Travel Planner
• Chatbot 🤖
  • bot to chat about the PDFs
• Travel Planner 🗺️
  • to create travel plans based on the PDFs

Decisions for Simplification

• no system prompt management in Pocketbase, everything is hardcoded in LLM Service
• one LLM (OpenAI) for chains
• one LLM (OpenAI) for embeddings
• file storage by using local file system
• chat history/session is stored in memory in LLM Service

Open Questions

• which database to use for LLM Database?
  • IDEA: Postgres with pgvector
  • https://github.com/pgvector/pgvector
  • Docker support: https://github.com/pgvector/pgvector
  • Langchain support: https://js.langchain.com/docs/integrations/vectorstores/pgvector/