search

nickthecook / archyve

GNU Affero General Public License v3.0
128 stars 15 forks source link

readme

README

Archyve is a web app that makes pretrained LLMs aware of a user's documents, while keeping those documents on the user's own devices and infrastructure.

Feature Tracker

Features go through these phases:

RAG KnowledgeGraph OpenAI Support Ollama/OpenAI Proxy Port
✅ Stable 🧪 Experimental ✅ Stable 🚧 In Development

Overview

Archyve enables Retrieval-Augmented Generation (RAG) by providing an API to query the user's docs for relevant context. The Archyve client provides Archyve with the prompt the user entered, and Archyve will return relevant text chunks that the client can include in its prompt to an LLM server.

Archyve has an experimental Knowledge Graph feature (a.k.a "Graph RAG").

Archyve provides:

For LLM servers, Archyve supports:

The Getting Started section will walk you through setting up Archyve for use with Ollama.

Screenshots

Chat UI

image

Documents UI

image

Graph database view

image

Concepts

Getting started

Running an Archyve container

To run Archyve, use docker compose.

  1. Clone this repo
  2. cp dotenv_template local.env
  3. Run openssl rand -hex 64 and put the value in the SECRET_KEY_BASE variable in your local.env file
  4. Run the container
docker compose up --build

If you see "✘ archyve-worker Error", don't worry about it. Docker will build the image and run it.

If you see an error like "failed to solve: error from sender: open /home/.../archyve/deps/postgres: permission denied", it's because Docker has a bug on linux where it will still try to traverse directories under the build dir even if they're in the .dockerignore file.

Work around this with mv deps ../archyve-deps. You'll need to mv ../archyve-deps deps when you want to run the app on your host again.

  1. Browse to http://127.0.0.1:3300 and log in with admin@archyve.io / password (you can change these values by setting USERNAME and PASSWORD in your local.env file and restarting the container)

WARNING: The container will write a file with local encryption keys into config/local. If you lose this file, the application will not be able to decrypt sensitive data within the database (e.g. passwords or API keys), and the database will need to be reset, losing all data.

If you want to migrate your database elsewhere, migrate this file along with it.

Running Archyve directly on your host machine

This section is about running Archyve directly on your host machine.

If you want to develop Archyve you probably want the app runing directly on your host, rather than in a container, to reduce the time it takes to try your changes.

Installing dependencies on a Mac

  1. Ensure that you have brew installed

  2. Ensure that you have docker set up and a "machine" configured and ready to pull and run container images

  3. Ensure you have ops installed:

brew install nickthecook/crops/ops

  1. Install Ollama and make sure you're running ollama serve and that you have the minimum models installed (see section on Ollama further below).

Installing dependencies on Linux

  1. Ensure that you have Docker and the appropriate compose plugins installed
  2. Ensure you have ops installed (download the binary for your platform).
  3. Install Ollama and make sure you're running ollama serve and that you have the minimum models installed (see section on Ollama further below).

Running Archyve

  1. Clone this repo
  2. cp config/dev/config.sample.json config/dev/config.json
  3. Edit config/dev/config.json, running the given commands and replacing them in the file with their output
  1. ops up
  2. ops rails db:setup (after the initial setup)
  3. ops rails neo4j:migrate
  4. ops server
  5. Open http://127.0.0.1:3300/ in a browser and you can login using admin@archyve.io and password.

ops loads environment variables from config/dev/config.json and config/dev/secrets.ejson. It loads actions and dependencies from ops.yml in your working directory. It will save you time.

You may want to alias rails="ops rails" if you're a Rails dev, and the rails ... muscle memory is hard to change.

There are more details on how to deploy Archyve to production.

Using Archyve

Archyve provides a web interface with a few different areas, focused on different things:

You can break things in the admin UI. When starting out, it's best just to use it to configure ModelServers, ModelConfigs, and Settings.

Document ingest

Documents uploaded to any Collection go through a few stages:

  1. Chunking (splitting the document into chunks of 1000 characters)
  2. Embedding (vectorizing the chunks using an embedding model)
  3. Storing in the vector database (ChromaDB)

If a stage takes longer than a second, the progress is shown in the UI. Once a document is in the Embedded state, you can test searching it to see what chunks would be returned for a given chat prompt.

When the Collection has the Knowledge Graph feature enabled, Documents also go through:

  1. Entity extraction (having an LLM identify entities and relationships from the text)

Then the Collection will go through a few stages:

  1. Entity summarization (creating a summary for each entity based on references found in the text during stage 4)
  2. Vectorization (vectorizing the summary using an embedding model and storing in ChromaDB)
  3. Graphing (pushing entities and relationships to Neo4j)

Viewing extracted entities

Once you have a document in the Extracted state, you can view the most commonly references entities that were extracted from Documents in a Collection. Click the "Top 10 entities by occurrences" tab on the Collection page to see the 10 entities in that Collection that appeared more often in Documents than any others.

Relationships are not yet visible in the Archyve UI, although you are presented with a count of the number of relationships each entity has.

Viewing the Knowledge Graph

Once you have a Collection in the Graphed state, you can view the knowledge graph in the Neo4j web interface:

You can query the graph using Cypher queries. E.g., for a Collection called "Greek Mythology", use this query to show the complete graph, with all relationships:

match (n:Nodes::Entity{collection_name: "Greek Mythology"}) return (n)

Archyve doesn't do anything with data once it's in Neo4j, but it will, and the Neo4j interface can be useful in assessing the quality of entity extraction.

API

See archyve.io for details on the ReST API.

Authentication

Archyve provides a ReST API. To use it, you must have:

  1. a Client ID (goes in the X-Client-Id header in all API requests)
  2. an API key (goes in the Authorization header after Bearer)

TODO: add this to the UI

If you are running the app on your host, you can set the DEFAULT_API_KEY and DEFAULT_CLIENT_ID environment variables. On startup, Archyve will ensure that a client with these credentials exists in its database.

If you are running the app via docker compose, set the above two environment variables in your local.env file and restart the containers.

If you are running the app on your host, set the two above environment variables in config/dev/config.json and run rails db:seed.

Sending authenticated requests

You should be able to send API requests like this:

curl -v localhost:3300/v1/collections \
  -H "Accept: application/json" \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "X-Client-Id: <YOUR_CLIENT_ID>"

See archyve.io for more information on the API.

See the next section for setting up Ollama for use by Archyve or document uploads and chat will fail.

If you are running Archyve directly on your host, you will have ops set up and you can use ops request to send requests to the Archyve API. E.g.:

$ ops request search `q=hello`
$ ops request collections/1/entities/1
$ ops request version

It will handle setting the authorization header for you.

Dependency details

Ollama

You can run a dedicated instance of Ollama in a container by adding it to the compose.yaml file, but it takes a while to pull a chat model, so the default here is to assume you already have an Ollama instance.

Archyve will use a local instance of Ollama by default. Ensure you have Ollama installed and running (with ollama serve) and then run the following commands to set up your Ollama instance for Archyve:

You can change which models Archyve will use in the Admin UI (/admin) under Settings in the menu on the left.

Keep in mind that whatever models you have Archyve use, you will need to pull those models in Ollama yourself (for now).

Embedding model

Archyve uses the nomic-embed-text model for all embeddings. Please make sure it is present in your Ollama server.

Changing the embedding model means you will need to delete all your Collections and re-ingest them, as all the embeddings will have changed and similarity search will not function. Archyve has not been testing with embedding models other than nomic-embed-text.

Summarization model

The summarization model is the model Archyve will use to summarize chats so it can set a brief title for each one, based on the first message in the chat.

phi3 is fast and usually decent at this, but you can change this model in the Admin UI (/admin) under Settings -> "summarization_model".

Entity extraction model

If you enable the Knowledge Graph for any Collections, Archyve will default to using llama3.1:8b as the entity extraction model. If you change this, you may get poor results from the KG. However, I'd be very interested to hear your experience if you do try another model!

OpenAI

Archyve works with OpenAI, but you need to provision one ModelServer and one ModelConfig first.

ModelServer

Go to "Admin" using the button in the bottom left, then click "ModelServers". Click the "+" in the top right of the ModelServer list and enter this info:

Click "Save".

ModelConfig

From the Admin UI, click ModelConfigs on the left. Click the "+" in the top right of ModelConfig list and enter this info:

Leave "Embedding" unchecked - this is not an embedding model. Click "Save".

You can now select this Model from the drop-down in a Conversation. To make this your default generation model, go to the ModelConfigs list in the Admin UI, click the three dots menu on the right side of your model config, then click "Use for chat".

OpenAI embeddings

These are not supported at the moment. Archyve needs to support the OpenAI embedding response format, and it needs to make it easy for users to use multiple embedding models at the same time, and make it clear what the limitations are of doing that.

Azure OpenAI

Archyve works with OpenAI, but you need to provision one ModelServer and one ModelConfig first.

ModelServer

Go to "Admin" using the button in the bottom left, then click "ModelServers". Click the "+" in the top right of the ModelServer list and enter this info:

Click "Save".

ModelConfig

From the Admin UI, click ModelConfigs on the left. Click the "+" in the top right of ModelConfig list and enter this info:

Leave "Embedding" unchecked - this is not an embedding model. Click "Save".

You can now select this Model from the drop-down in a Conversation. To make this your default generation model, go to the ModelConfigs list in the Admin UI, click the three dots menu on the right side of your model config, then click "Use for chat".

Azure OpenAI embeddings

These are not supported at the moment. Archyve needs to support the OpenAI embedding response format, and it needs to make it easy for users to use multiple embedding models at the same time, and make it clear what the limitations are of doing that.

Admin UI

There is an admin UI running at http://127.0.0.1:3300/admin. There, you can view and change Settings, ModelConfigs, and ModelServers if you are logged in as an admin.

There is a link to it in the bottom of the side bar if you are logged into Archyve as an admin.

Jobs

Archyve uses a jobs framework called Sidekiq. It has a web UI that you can access at http://127.0.0.1/sidekiq if you are logged in as an admin.

Knowledge Graph

Archyve uses the term "Knowledge Graph" instead of Graph RAG because "RAG" is ambiguous. Everything is "retrieved" from somwhere.

Archyve has an experimental Knowledge Graph (KG) feature (a.k.a. "Graph RAG"). The user enables the feature per-Collection. It is safe to use, but will generate orders of magnitude more calls to your LLM server than not using it.

If you enable KG in any Collections, it is highly recommended that you use llama3.1 as your entity_extraction_model. Many other models are simply not able to extract entities in a meaningful way with the prompts used by Archyve.

This feature is a work-in-progress, but it seems effective at providing relevant context at the moment. It will extract entities and the relationships between those entities from the text of your documents. It will then create a summary of each entity and store that in the vector database. These entity summaries will be returned by Archyve API search along with relevant chunks of your document.

If a Collection has KG enabled, chatting with Archyve about that Collection will use the KG to augment your prompts.

Archyve KG is based on Microsoft's Graph RAG project. Archyve is as far as extracting entities and relationships from uploaded documents, but does not perform community detection or entity deduplication yet.