Add Budget Manager + Support for Anthropic, Cohere, Palm (100+ LLMs using LiteLLM)

[ishaan-jaff]

Addressing: https://github.com/StampyAI/stampy-chat/issues/47 https://github.com/StampyAI/stampy-chat/issues/55

This PR addresses two problems:

• Add support for 100+ LLMs
• Use a budget manager for limiting $ spend per session or per user

Add support for 100+ LLMs

using LiteLLM https://github.com/BerriAI/litellm/ LiteLLM is a lightweight package to simplify LLM API calls - use any llm as a drop in replacement for gpt-3.5-turbo.

Example

from litellm import completion
## set ENV variables
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"
messages = [{ "content": "Hello, how are you?","role": "user"}]
# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages)

Use a budget manager for limiting $ spend per session or per user

LiteLLM exposes a budget manager for each session/user

• We init a budget manager
• Check the budget manager before making completion calls
• Update with costs after making completion calls

[ishaan-jaff]

@FraserLee @henri123lemoine can i get a review on this pr ?

if this initial commit looks good i can add docs/testing too