config-template.yaml
AOE, an acronym from DOTA2 for Area Of Effect, denotes an ability that can affect a group of targets within a certain area. Here, AOE in AI implies that user can obtain parallel outputs from multiple LLMs with one single prompt at the same time.
Currently, there are many open-source frameworks based on the ChatGPT for chat, but the LGC(LLM Group Chat) framework is still not coming yet.
The emergence of OpenAOE fills this gap: OpenAOE can help LLM researchers, evaluators, engineering developers, and even non-professionals to quickly access the market's well-known commercial and open-source LLMs, providing both single model serial response mode and multi-models parallel response mode.
OpenAOE can:
[!TIP] Require python >= 3.9
We provide three different ways to run OpenAOE: run by pip
, run by docker
and run by source code
as well.
pip install -U openaoe
openaoe -f /path/to/your/config-template.yaml
There are two ways to get the OpenAOE docker image by:
pull the OpenAOE docker image
docker pull opensealion/openaoe:latest
or build a docker image
git clone https://github.com/internlm/OpenAOE
cd OpenAOE
docker build . -f docker/Dockerfile -t opensealion/openaoe:latest
docker run -p 10099:10099 -v /path/to/your/config-template.yaml:/app/config.yaml --name OpenAOE opensealion/openaoe:latest
git clone https://github.com/internlm/OpenAOE
cd OpenAOE/openaoe/frontend
npm install
npm run build
cd OpenAOE # this OpenAOE is the clone directory
pip install -r openaoe/backend/requirements.txt
python -m openaoe.main -f /path/to/your/config-template.yaml
[!TIP]
/path/to/your/config-template.yaml
is a configuration file loaded by OpenAOE at startup, which contains the relevant configuration information for the LLMs, including: API URLs, AKSKs, Tokens, etc. A template configuration yaml file can be found inopenaoe/backend/config/config-template.yaml
. Note that, thisconfig-template.yaml
DOES NOT contain any API access data, you should add them by yourself.
You are always welcome to fork this project to contribute your work and find the TODOs in furture.
If you want to add more LLMs' APIs or features based on OpenAOE, the following info might be helpful.
The technology stack we use includes:
[!TIP] The build tools can be installed quickly by
pip install -U sealion-cli
openaoe/frontend
openaoe/backend
openaoe/main.py
name
, avatar
, provider
, etc in openaoe/frontend/src/config/model-config.ts
openaoe/frontend/src/config/api-config.ts
openaoe/frontend/src/services/fetch.ts
, you may need to change the payload structure and handle corner cases according to your model's API definition.