Create an initial architecture

[Tyler-R-Kendrick]

Initial Architecture for Graph DB Enhanced RAG Solution

Overview

The purpose of this feature request is to design the initial architecture for a graph database (Graph DB) enhanced RAG solution for AI search. The solution will leverage various Azure services to achieve the desired functionality.

Key Resources

1. Azure AI Search

• Utilize Azure AI Search for hybrid search capabilities.
• Implement both semantic search and keyword search functionalities.
• Integrate with other Azure services to enhance search results.

2. Azure AI Services (AOAI)

• Incorporate GPT-4o (part of Azure AI Services) for natural language understanding and generation.
• Leverage AOAI to enhance the quality of answers and responses.

3. Azure App Services

• Develop a Python API using FastAPI.
• Apply guidance, guardrails, and LangChain to ensure secure and efficient API development.
• Host the API on Azure App Services.

4. Azure Storage

• Utilize Azure Storage for document storage and app service storage.
• Store documents that will be indexed and searched by Azure AI Search.

5. Azure SQL Servers (Graph DB)

• Implement a graph database using Azure SQL Servers.
• Model relationships between entities and enable efficient querying for AI search.

Pluggable Architecture

• Design the API host with a pluggable architecture.
• Allow for easy swapping of Azure-specific services (e.g., switching to a different search engine or database) without major code changes.

Agent Architecture

• One agent to conduct classic RAG.
• One agent to conduct enhanced graph RAG.

Next Steps

The proposed architecture will serve as the foundation for the RAG solution. Detailed design, implementation, and testing will follow based on this initial plan.

Please feel free to provide feedback or additional requirements.

[Tyler-R-Kendrick]

Based on our conversations so far, this is my understanding of a first pass at the initial architecture.

graph TD
  subgraph AzureServices
    search[Azure AI Search]
    AOAI[Azure AI Services]
    app[Azure App Services]
    storage[Azure Storage]
    sqlGraph[Azure SQL Servers]
  end

  subgraph Agents
    rag[Classic RAG Agent]
    graphRag[Enhanced Graph RAG Agent]
  end

  rag --> search --> storage --> app --> AOAI
  graphRag --> sqlGraph --> app --> AOAI
  app --> rag --> graphRag --> AOAI

sequenceDiagram
  autonumber
  participant rag as RAG Agent
  participant graphRag as Graph RAG Agent
  participant search as Azure AI Search
  participant AOAI as Azure AI Services
  participant app as Azure App Services
  participant storage as Azure Storage
  participant sqlGraph as Azure SQL Servers

  user ->> app: prompt for information
  app ->> +rag: invoke rag agent to enhance prompt
  rag ->> search: grab vectorized content through semantic search.
  search ->> storage: search through indexed content on storage
  storage-->>search: return relevant indexed documents
  search-->>rag: return relevant document chunks/partitions.
  rag-->>app: return enhanced prompt
  deactivate rag

  app ->> +graphRag: invoke graphRag agent to further enhance prompt.
  graphRag ->> sqlGraph: query relations from entity edges of rag retrieved entities
  sqlGraph-->>graphRag: return names of edges.
  graphRag ->> AOAI: request to filter related entities and generate a query from user prompt.
  AOAI-->>graphRag: return a structure query for execution.
  graphRag ->> sqlGraph: search with AOAI generated graph query
  sqlGraph-->>graphRag: return relevant entities.
  deactivate graphRag

  app ->> AOAI: send prompt enhanced by classic and graph enhanced RAG.
  AOAI-->app: return LLM response
  app -->> app: sanitize LLM response for correctness
  app-->>user: return sanitized response

[Tyler-R-Kendrick]

Attempt at sequence flow for agent to llm and graph db.

sequenceDiagram
    autonumber
    actor u as User
    participant a as Agent
    participant ai as LLM
    participant db as Graph Database

    activate a
    a->>db: query graph ontology
    activate db
    db-->>a: return nodes and edges
    deactivate db
    a->>a: Serialize ontology as rdf.
    a->>a: Populate system prompt with serialized ontology
    activate u
    u->>a: Provide initial user prompt
    critical Get user intent
      a->>ai: Identify user intent from user prompt
      activate ai
      ai-->>a: Return intent as goal
      deactivate ai
    option Plan
      a->>ai: Break down prompt into execution plan for goal
      activate ai
      ai-->>a: Return execution plan
      deactivate ai
    option Generate graph query
      a->>ai: Generate graph query based on execution plan
      activate ai
      ai-->>a: return graph query for target language.
      deactivate ai
    option Execute graph
      a->>db: Execute graph query
      activate db
      db-->>a: Return graph query results
      deactivate db
      a->>a: Augment prompt with graph response
    end
    a-->>ai: submit graphRAG enhanced user prompt to LLM
    activate ai
    ai-->>a: return response to user prompt
    deactivate ai
    a-->>u: return response to user    
    deactivate a
    deactivate u

[Tyler-R-Kendrick]

I added an architecture document located here: https://github.com/microsoft/deepRAG/blob/main/docs/ARCHITECTURE.md

[Tyler-R-Kendrick]

should we consider topic extraction for both the user prompt and the response? Or just the initial RAG response from the LLM?

[Tyler-R-Kendrick]

initial proposal for api schema:

openapi: 3.0.0
info:
  title: RAG API
  description: API for performing VectorRAG and GraphRAG operations
  version: 1.0.0
paths:
  /vector-rag:
    post:
      summary: Execute VectorRAG
      description: Takes a user request and returns a response using VectorRAG method.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                userRequest:
                  type: string
                  example: "Your request here"
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  response:
                    type: string
                    example: "Your response here"
        '400':
          description: Bad request
  /graph-rag:
    post:
      summary: Execute GraphRAG
      description: Takes a user request and returns a response using GraphRAG method.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                userRequest:
                  type: string
                  example: "Your request here"
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  response:
                    type: string
                    example: "Your response here"
        '400':
          description: Bad request