[Feature]: Add design policy document for huge utils.py maintenance

[nobu007]

The Feature(request)

How about to add design policy document? It makes easy to maintenance.

Content idea

The role of variables

model

The model name. For example. https://litellm.vercel.app/docs/providers/openrouter

custom_llm_provider

Provider name. Like openai, anthropic, openrouter.

In default, it is the first item of model. anthropic/claude-3-xx => anthropic openrouter/anthropic/claude-3-xx => openrouter

api_base

dynamic_api_key

Overall of decision flow

Priority order is here.

1. model
2. custom_llm_provider
3. api_base
4. dynamic_api_key

Decision flow of each variables

(It may over detail as design.)

model

1. Input from parameter.

custom_llm_provider

1. Input from parameter.
2. If not set, xxx

api_base

1. Input from parameter.
2. If not set, xxx

dynamic_api_key

1. llm_provider decides this.

Note: Why this variable exist? When should you use this?

Motivation

I have a questions about here. https://github.com/BerriAI/litellm/blob/main/litellm/utils.py

• What is custom_llm_provider? How can I decide to add new custom_llm_provider?
• The decision logic of model, api_base, dynamic_api_key is unclear.
• How to initialize xx_models(like anthropic_models).

Memo for refactoring in future

get_llm_provider

This function decides "custom_llm_provider", "dynamic_api_key", "api_base".

TODO:

• Remove arg "api_key".

• Remove retun value "dynamic_api_key"

• Add notice comment for replacement api_base -> base_url. Ref: https://github.com/microsoft/autogen/pull/383/files

[ishaan-jaff]

contributions welcome on this