Add support for Zephyr 7B

OpenBMB / ChatDev

Create Customized Software using Natural Language Idea (through LLM-powered Multi-Agent Collaboration)

https://arxiv.org/abs/2307.07924

Apache License 2.0

25.57k stars 3.21k forks source link

Add support for Zephyr 7B #295

Open gvijqb opened 11 months ago

gvijqb commented 11 months ago

Zephyr 7B model is kind of the best in small size models and is able to outperform GPT 3.5 and Llama 70B models as well in some tasks. See benchmark: https://huggingface.co/HuggingFaceH4/zephyr-7b-beta

It is also cost effective because of its low size.

Zephyr 7B API is available on MonsterAPI: Free playground: https://monsterapi.ai/playground/zephyr-7b-beta Zephyr API docs: https://developer.monsterapi.ai/reference/post_generate-zephyr-7b-beta

We can provide dedicated support for ensuring scalable and reliable integration.

zwilch commented 10 months ago

you are responsible for your LLM

start your onw llm for example use oobabooga textgen webui with parameter "--listen --extension openai" load your Zephyr Model in oobabooga

ChatDev to use your own model

write the commands in ChatDev console

export OPENAI_API_KEY="no key needed in oobabooga"
export OPENAI_API_BASE="http://YOUR-OOBABOOGA-IP/v1"

pythron run.py --tast "build a HTML one page with a clock" --name "my-oobabooga-zephyr-clock"

Liagim01 commented 10 months ago

You could refer to #27 to get an insight on how to deploy local models into ChatDev.

linussmashtips commented 9 months ago

ok so running it with LMstudio and openai version 1.3.3 you get an authentication error so i changed OPENAI_API_BASE to BASE_URL and it connected to LM studio but now i was getting the "messages" error referenced here. Switched it to llama2 and now it's working. cant wait to test this out.