search

Azure-Samples / contoso-chat

This sample has the full End2End process of creating RAG application with Prompty and AI Studio. It includes GPT 3.5 Turbo LLM application code, evaluations, deployment automation with AZD CLI, GitHub actions for evaluation and deployment and intent mapping for multiple LLM task mapping.
MIT License
468 stars 2.88k forks source link

Some default options are $expensive$ for a low-throughput example program #53

Closed nickcoast closed 9 months ago

nickcoast commented 9 months ago
  1. Default for Cosmos DB is Provisioned throughput. Serverless should be much cheaper for this sample program.
  2. Default for Prompt Flow deployment is 3 instances. Would 1 work? What about different instance types?

Question If I create a new Cosmos DB using Serverless, can I swap it out with my current provisioned DB? Can I just create a new connection for it in Azure and update .env in the Codespace. Or I could start over with the updated repo and edit the bicep file section for Cosmos DB, but that seems like a lot more work.

cassiebreviu commented 9 months ago

You can choose the SKU and instances based on the needs your application. So you are able to scale down to 1 instance if that is all that is needed.

You can use any datasource you prefer. We are using cosmosdb in this sample but you could always update to a different data resource.

This sample is meant to show how to do things and be a starting point. You could and should update based on the requirements of your application and scenario.