Add Claude 3 and other models to the config.toml

[PierrunoYT]

Add Claude 3 and other models to the config.toml

[scarr05]

LLM_MODEL="claude-3-haiku-20240307" and LLM_API_KEY="sk-xxx"

worked for me, however the Claude API's have pretty tight rate limits so it was mostly unusable with OpenDevin for any coding tasks.

Claude on bedrock works a treat.

[rbren]

See https://github.com/OpenDevin/OpenDevin/issues/332 for rate limiting

[djaym7]

LLM_MODEL="claude-3-haiku-20240307" and LLM_API_KEY="sk-xxx"

worked for me, however the Claude API's have pretty tight rate limits so it was mostly unusable with OpenDevin for any coding tasks.

Claude on bedrock works a treat.

how did you run it on bedrock ? what does your command to run docker look like ?

[djaym7]

i'm running this but running into all kinds of error

[djaym7]

using main branch works fine, but can someone please add requirement boto3 ?

Oops. Something went wrong: BedrockException - Traceback (most recent call last): File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock.py", line 712, in completion client = init_bedrock_client( ^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock.py", line 575, in init_bedrock_client import boto3 ModuleNotFoundError: No module named 'boto3'

[scarr05]

the error your seeing suggests that boto python package isnt installed into the docker instance ?

I got this running on an earlier version of Open Devin, not sure if its changed since but here is a snip of my config.toml

I will see if i can give it a go tomorrow with the latest and see how it runs, but from looking at the repo this has changed quite a bit since when i was testing 3 weeks ago

[scarr05]

your settings look like its passing the correct env variables through to the container, so i suspect its because of the missing package, also check the version of boto3 on the container in case this is an old one from before the bedrock runtime was available.

Hope this helps, will update tomorrow once i have some time to play.

[djaym7]

thanks, this worked for me :

docker run \ -e AWS_ACCESS_KEY_ID \ -e AWS_SECRET_ACCESS_KEY \ -e AWS_REGION_NAME \ -e LLM_MODEL="bedrock/anthropic.claude-3-sonnet-20240229-v1:0" \ -e WORKSPACE_MOUNT_PATH=$WORKSPACE_DIR \ -v $WORKSPACE_DIR:/opt/workspace_base \ -v /var/run/docker.sock:/var/run/docker.sock \ -p 3000:3000 \ -e SANDBOX_TYPE=exec \ -v $HOME/.aws:/root/.aws \ -v /home/Agent/llm_lite_bedrock.py:/tmp/llm_lite_bedrock.py \ ghcr.io/opendevin/opendevin:main \ /bin/bash -c "pip install boto3 && \ cp /tmp/llm_lite_bedrock.py /app/.venv/lib/python3.12/site-packages/litellm/llms/bedrock.py && \ uvicorn opendevin.server.listen:app --host 0.0.0.0 --port 3000"