Local Models integration

[TheWhiteWord]

Hey Devs, let me start by saying that this programme is great. Well done on your work, and thanks for sharing it.

My question is: is there any plan to allow for the integration of local models? Even a just section in the documentation would be great.

Have a good day theWW

[resdig3]

I'm just going to add a bit about how I got ChatDev running locally with LM Studio server for anyone searching. It was really easy if there would have been clear instructions but I had to read through all of the issues and tried to find stuff in the code to no avail.

Anyway. The basics:

• Windows 10
• Following the installation instructions from the readme for steps 1-3 (gitclone, conda, cd, install requirements)

On step 4 do this instead:

set OPENAI_API_BASE=http://localhost:1234/v1

And that's it (you'll need to start the LMS server and load a model), now you can just run ChatDev like you normally would but locally.

Trying to get this running on a Win10 machine. I keep getting this error, like it needs a working API key of some sort:

.conda\envs\ChatDev_conda_env\lib\site-packages\openai_base_client.py", line 877, in _request raise self._make_status_error_from_response(err.response) from None openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: abc123xyz. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

[vishusinghal]

FYI - its not OPEN_API_BASE. if using anaconda on windows you do SET BASE_URL="http://localhost:1234/v1" and then SET OPEN_API_KEY="not needed" . this is if you're using LMstudio. All working my end using Mistral instruct 7B.

I tried with BASE_URL as well as OPEN_AI_BASE but getting an APIConnectionError

tenacity.RetryError: RetryError[<Future at 0x1cdcf0bce80 state=finished raised APIConnectionError>]

Can you help?

[maramowicz]

It's not BASE_URL, it's OPENAI_BASE_URL! But I also saw on begin that OPENAI_API_BASE is also used (maybe I'm wrong here but it works for me), so the correct command is OPENAI_API_BASE=http://localhost OPENAI_BASE_URL=http://localhost OPENAI_API_KEY="anything" python run.py --task "Snake game in pure html" --name "WebSnake"

[hemangjoshi37a]

hi all the contributors , I have found a github app that helps solving issues using AI and the approach to this is very interesting so i am sharing this , please anyone add this to this repo . : https://github.com/apps/sweep-ai not sponsored or anything but i found it helpful so.

[BradKML]

@acbp has anyone wrote documentation specifically for things like Ollama (or LocalAI, Xorbit, or OpenLLM) to work with ChatDev? LiteLLM maybe?

[hemangjoshi37a]

If anyone has achieved or can suggest for this . In this we want to continuously index the updated code base in the HippoRAG index and query on the updated index and then make code changes and continuously do so. Here consider that we want to do this offline with ollama type models only and dont want to use OpenAI or Claude .

If anyone can suggest how can i do this ?

[hgftrdw45ud67is8o89]

anyone had success with llama.cpp?

I got tenacity.RetryError: RetryError[<Future at 0x2595e8f1e10 state=finished raised NotFoundError>]

and

raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: Error code: 404 - {'error': {'code': 404, 'message': 'File Not Found', 'type': 'not_found_error'}}

[syedshahzebhasnain]

On Mac( m3) i used this with local install ( Anaconda)

BASE_URL=http://localhost:1234/v1 OPENAI_API_KEY=dummy python3 run.py --task "< task >" --name "