search

anarchy-ai / LLM-VM

irresponsible innovation. Try now at https://chat.dev/
https://anarchy.ai/
MIT License
465 stars 150 forks source link
artificial-intelligence deep-learning distillation distillation-model llm llm-agent llm-inference llm-local llm-training machine-learning

readme

Anarchy Logo

🤖 Anarchy Labs' LLM-VM 🤖

An Open-Source AGI Server for Open-Source LLMs

🛑 DEVELOPMENT ON PAUSE 🛑

This is Anarchy's effort to build 🏗️ an open generalized artificial intelligence 🤖 through the LLM-VM: a way to give your LLMs superpowers 🦸 and superspeed 🚄.

You can find detailed instructions to try it live here: anarchy.ai

This project is in BETA. Expect continuous improvement and development.

Table of Contents

📚 About 📚

💁 What is the Anarchy LLM-VM?

The Anarchy LLM-VM is a highly optimized and opinionated backend for running LLMs with all the modern features we've come to expect from completion: tool usage, persistent stateful memory, live data augmentation, data and task fine-tuning, output templating, a web playground, API endpoints, student-teacher distillation, data synthesis, load-balancing and orchestration, large context-window mimicry.

Formally, it is a virtual machine/interpreter for human language, coordinating between data, models (CPU), your prompts (code), and tools (IO).

By doing all these things in one spot in an opinionated way, the LLM-VM can properly optimize batch calls that would be exorbitantly expensive with distributed endpoints. It furthermore strives for both model and architecture agnosticism, properly optimizing the chosen model for the current architecture.

🤌 Why use the Anarchy LLM-VM?

In line with Anarchy's mission, the LLM-VM strives to support open-source models. By utilizing open-source models and running them locally you achieve several benefits:

🎁 Features and Roadmap

🚀 Quickstart 🚀

🥹 Requirements

Installation Requirements

Python >=3.10 Supported. Older versions of Python are on a best-effort basis.

Use bash > python3 --version to check what version you are on.

To upgrade your python, either create a new python env using bash > conda create -n myenv python=3.10 or go to https://www.python.org/downloads/ to download the latest version.

 If you plan on running the setup steps below, a proper Python version will be installed for you

System Requirements

Different models have different system requirements. Limiting factors on most systems will likely be RAM, but many functions will work at even 16 GB of RAM.

That said, always lookup the information about the models you're using, they all have different sizes and requirements in memory and compute resources.

👨‍💻 Installation

The quickest way to get started is to run pip install llm-vm in your Python environment.

Another way to install the LLM-VM is to clone this repository and install it with pip like so:

> git clone https://github.com/anarchy-ai/LLM-VM.git
> cd LLM-VM
> ./setup.sh

The above bash script setup.sh only works for MacOS and Linux.

Alternatively you could do this:

> git clone https://github.com/anarchy-ai/LLM-VM.git
> cd LLM-VM
> python -m venv <name>
> source <name>/bin/activate
> python -m pip install -e ."[dev]"

If you are on Windows. You can follow either of the below two methods:

Before doing any of the following steps, you have to first open Powershell as administrator and run the below command

> Set-ExecutionPolicy RemoteSigned
> Press Y and enter
> exit

Now you can follow either of the below two methods:

  1. Open Powershell and do this:
> git clone https://github.com/anarchy-ai/LLM-VM.git
> cd LLM-VM
> .\windows_setup.ps1

or

  1. Open Powershell and do this:
> winget install Python.Python.3.11
> python --version
> git clone https://github.com/anarchy-ai/LLM-VM.git
> cd LLM-VM
> python -m venv anarchyai
> anarchyai\Scripts\activate
> python -m pip install -e .

Note:

  1. For the above steps to work you have to be on Windows 10 1709 (build 16299) or later build.
  2. Enable developer mode in windows settings(not compulsory but if enabled will give an added advantage)

One Last Step, almost there!

If you're using one of the OpenAI models, you will need to set the LLM_VM_OPENAI_API_KEY environment variable with your API key.

✅ Generating Completions

Our LLM-VM gets you working directly with popular LLMs locally in just 3 lines. Once you've installed it (as above), just load your model and start generating!

# import our client
from llm_vm.client import Client

# Select which LLM you want to use, here we have OpenAI 
client = Client(big_model = 'chat_gpt')

# Put in your prompt and go!
response = client.complete(prompt = 'What is Anarchy?', context = '', openai_key = 'ENTER_YOUR_API_KEY')
print(response)
# Anarchy is a political ideology that advocates for the absence of government...

🏃‍♀ Running LLMs Locally

# import our client
from llm_vm.client import Client

# Select the LlaMA 2 model
client = Client(big_model = 'llama2')

# Put in your prompt and go!
response = client.complete(prompt = 'What is Anarchy?', context = '')
print(response)
# Anarchy is a political philosophy that advocates no government...

😎 Supported Models

Select from the following models


Supported_Models = ['chat_gpt','gpt','neo','llama2','bloom','opt','pythia']

☯ Picking Different Models

LLM-VM default model sizes for local models are intended to make experimentation with LLMs accessible to everyone, but if you have the memory required, larger parameter models will perform far better!

for example, if you want to use a large and small neo model for your teacher and student, and you have enough RAM:

# import our client
from llm_vm.client import Client

# Select the LlaMA model
client = Client(big_model = 'neo', big_model_config={'model_uri':'EleutherAI/gpt-neox-20b'}, 
                small_model ='neo', small_model_config={'model_uri':'EleutherAI/gpt-neox-125m'})

# Put in your prompt and go!
response = client.complete(prompt = 'What is Anarchy?', context = '')
print(response)
# Anarchy is a political philosophy that advocates no government...
Here are some default model's details: Name Model_Uri Model params Checkpoint file size
Neo EleutherAI/gpt-neo-1.3B 1.3B 5.31 GB
Bloom bigscience/bloom-560m 1.7B 1.12 GB
OPT facebook/opt-350m 350m 622 MB

For some other choices of memory usage and parameter count in each model family, check out the tables model_uri_tables.

🛠 Tool Usage

There are two agents: FLAT and REBEL.

Run the agents separately by going into the src/llm_vm/agents/<AGENT_FOLDER> and running the file that is titled agent.py.

Alternatively, to run a simple interface and choose an agent to run from the CLI, run the src/llm_vm/agents/agent_interface.py file and follow the command prompt instructions.

🩷 Contributing 🩷

We welcome contributors! To get started is to join our active discord community. Otherwise here are some ways to contribute and get paid:

Jobs

Bounty

We offer bounties for closing specific tickets! Look at the ticket labels to see how much the bounty is. To get started, join the discord and read the guide

🙏 Acknowledgements 🙏

License

MIT License