[FEAT]: AWS Bedrock temperature adjustment

Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.

https://anythingllm.com

MIT License

25.28k stars 2.56k forks source link

[FEAT]: AWS Bedrock temperature adjustment #2155

Closed ytjhai closed 2 months ago

ytjhai commented 2 months ago

What would you like to see?

I'm currently using the AWS Bedrock feature and it looks like bedrock supports adjusting the temperature through their API. It'd be helpful to have this in the settings since some models operate at different optimum temperatures than others (ex: Mistral large has a different "optimum" temperature than Llama 3.1)

timothycarambat commented 2 months ago

We do support temperature! https://github.com/Mintplex-Labs/anything-llm/blob/f7756d475812989ff3b78cd36b40250ed4539b3c/server/utils/AiProviders/bedrock/index.js#L179

You can set it per-workspace in the workspaces Chat Settings (scroll to bottom)

ytjhai commented 2 months ago

Ah I see it! Thanks for the quick response!