Bedrock Support

[dnlbreen]

Is your feature request related to a problem? Please describe. Yes, the problem is that the Devika project currently does not support integration with the Bedrock AI platform, which provides access to the Claude 3 language model. As mentioned in the project description, Devika utilizes Claude 3 for optimal performance. However, users who rely on the Bedrock platform for accessing Claude 3 cannot directly benefit from Devika's capabilities.

Describe the solution you'd like The solution I would like to see is the integration of Bedrock support into the Devika project. This would involve adding the necessary code and configurations to enable Devika to communicate with the Bedrock API and utilize the Claude 3 model hosted on the Bedrock platform. By supporting Bedrock, Devika would become accessible to a broader user base, including those who prefer or are required to use the Bedrock platform for various reasons, such as compliance, security, or organizational policies.

Describe alternatives you've considered One alternative I have considered is to continue using Devika with the existing OpenAI or Anthropic API integrations for accessing Claude 3. However, this approach may not be feasible or desirable for users who are already invested in the Bedrock ecosystem or have specific requirements that necessitate the use of Bedrock.

Another alternative would be to fork the Devika project and create a separate version that integrates with Bedrock. While this approach would work, it would lead to fragmentation and potentially duplicate efforts, making it harder to maintain and contribute to the project.

[obliviousz]

I'm guessing there might be some API key in bedrock too for claude? That doesn't work right?

[dnlbreen]

The Boto3 library handles authentication and authorization with AWS services like Bedrock Runtime using our AWS credentials and IAM permissions. No explicit API key is needed in the code. As long as we have the proper AWS setup, the provided code should work without an API key.

import boto3 import json

class Bedrock: def init(self): self.bedrock = boto3.client(service_name='bedrock-runtime', region_name="us-east-1")

def inference(self, prompt: str) -> str:
    prompt_config = {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 4096,
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": prompt},
                ],
            }
        ],
    }

    body = json.dumps(prompt_config)
    model_id = "anthropic.claude-3-sonnet-20240229-v1:0"
    accept = "application/json"
    content_type = "application/json"

    response = self.bedrock.invoke_model(
        body=body, modelId=model_id, accept=accept, contentType=content_type
    )
    response_body = json.loads(response['body'].read().decode('utf-8'))
    return response_body['content'][0]['text']