statefb
commented
2 months ago This sample is not intended for use by only 1-2 people. It is designed for multi-user deployments. The majority of the cost comes from the Aurora database. If you wish to reduce costs, please refer to the instructions at https://github.com/aws-samples/bedrock-claude-chat#stopping-vector-db-for-rag on how to stop the Aurora database when not in use.
Describe the solution you'd like
If the deployment is only being used a person or two, the cost of running the current infrastructure surrounding bedrock far exceeds both the charges billed to use the models, as well as other alternative products (ChatGPT). I would like to see a branch made that optimizes for cost so that people could deploy this as a self-hosted alternative to ChatGPT.
Why the solution needed
I've had the project up and running in a personal account and have enjoyed using it. But with zero modifications it's accrued ~$150USD in charges this month just for the surrounding resources. I've completed ~50 conversations accruing only a couple dollars. As someone who was looking to use this as a self-hosted alternative to ChatGPT, the benefits this project provides do not offset the monetary cost difference.
Additional context
Here you can see, I spun up the project towards the end of March and have had it running through April. If you then look at the breakdown you can see that bedrock doesn't even make it to the top 5 line items for the month, coming in at less than $5USD. With RDS and the nat gateway (EC2 - Other line item) being the lion's share.
Implementation feasibility
Are you willing to discuss the solution with us, decide on the approach, and assist with the implementation?