[Feature]: AmazonBedrock AI provider

k8sgpt-ai / k8sgpt

Giving Kubernetes Superpowers to everyone

http://k8sgpt.ai

Apache License 2.0

5.83k stars 677 forks source link

[Feature]: AmazonBedrock AI provider #716

Closed stevensu1977 closed 1 year ago

stevensu1977 commented 1 year ago

Checklist

[X] I've searched for similar issues and couldn't find anything matching
[X] I've discussed this feature request in the K8sGPT Slack and got positive feedback

Is this feature request related to a problem?

None

Problem Description

I use AWS EKS and want use Amazon Bedrock Claude model, so I fork repo and write it.

Solution Description

Amazon Bedrock AI provider

Benefits

If you use AWS EKS, you can use k8sgpt from the Amazon Bedrock AI provider, and your data will only be kept in the Amazon VPC

Potential Drawbacks

No response

Additional Information

No response

krrishdholakia commented 1 year ago

Hey @stevensu1977

Use LiteLLM for spinning up OpenAI-compatible server that translates openai/non-openai calls.

https://docs.litellm.ai/docs/proxy_server

litellm --model bedrock/anthropic.claude-instant-v1

# OPENAI Compatible Server running on http://0.0.0.0:8000

k8sgpt auth add --backend localai --model bedrock/anthropic.claude-instant-v1 --baseurl http://0.0.0.0:8000

stevensu1977 commented 1 year ago

Hi @krrishdholakia Thanks your comments , LiteLLM is good idea , it's a great project , but I just think if k8sgpt native support amazon bedrock that's be independently , like other k8s tools .