search

aws-samples / bedrock-access-gateway

OpenAI-Compatible RESTful APIs for Amazon Bedrock
MIT No Attribution
210 stars 43 forks source link

Support for Llama 3.1? #31

Open oemd001 opened 1 month ago

oemd001 commented 1 month ago

Hey, I was trying to call Llama 3.1 on the proxy, but I got a "model not supported" error. I was looking at the code and noticed that the entry for Llama 3.1 hasn't been added yet.

https://github.com/aws-samples/bedrock-access-gateway/blob/f6b73152bc18ac229f02472be6f5cf48bf303922/src/api/models/bedrock.py#L117

Is there a timeline in which this support could be added in?

daixba commented 1 month ago

Sorry, have been busy recently.

Llama 3.1 should be supported now, you can follow this https://github.com/aws-samples/bedrock-access-gateway#how-to-upgrade to start testing it.

Note that Tool call for Llama 3.1 is not yet supported by this project, I will need more tests.