Open caufieldjh opened 1 month ago
The litellm package tracks metadata on model context limits in this file: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json
litellm
This may also be retrieved and cached with their helper function:
from litellm import get_model_cost_map models = get_model_cost_map("") # This can take a URL but by default it uses the one above
models["gpt-4"] {'max_tokens': 4096, 'max_input_tokens': 8192, 'max_output_tokens': 4096, 'input_cost_per_token': 3e-05, 'output_cost_per_token': 6e-05, 'litellm_provider': 'openai', 'mode': 'chat', 'supports_function_calling': True}
The list includes multiple model types: completions, embeddings, etc.
The
litellm
package tracks metadata on model context limits in this file: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.jsonThis may also be retrieved and cached with their helper function:
The list includes multiple model types: completions, embeddings, etc.