DJAndries / llmvm

A protocol and modular application suite for language models.
Mozilla Public License 2.0
85 stars 4 forks source link

llmvm

GitHub

A protocol and modular application suite for language models.

Includes a code assistant that automatically retrieves context, powered by LSP.

Overview

llmvm consists of three types of executable applications:

The protocol acts as the glue between the above applications. Uses multilink and tower to achieve this.

Available crates

IPC details

Each component can interact with a dependency component via three methods:

This allows for some flexible hosting configurations. Here are some examples:

Hosting scenarios

Benefits

Installation

cargo is needed to install the binaries. Use rustup to install cargo.

Install the core by running:

cargo install llmvm-core

Install the desired frontends & backends listed under "Available crates". See their READMEs for more details.

Usage / configuration

See the README of each relevant component for more information on usage and configuration.

Model IDs

Model IDs in llmvm are strings consisting of three parts:

<backend name>/<provider name>/<model name>

The provider name must have the suffix -chat or -text

Examples:

By default, the core will invoke the process llmvm-<backend name> for local process communication.

Presets / Projects / Threads / Prompt Templates

See the core README for more information.

Model weights

See the relevant backend README (i.e. llmrs).

License

Mozilla Public License, version 2.0