search

AgentOps-AI / tokencost

Easy token price estimates for 400+ LLMs. TokenOps.
https://agentops.ai
MIT License
1.48k stars 61 forks source link

Dynamically load newest model prices #22

Closed silver5753 closed 10 months ago

silver5753 commented 10 months ago

https://github.com/AgentOps-AI/tokencost/blob/0eb8d23999bc12670643a89af2cf4be64747f668/tokencost/constants.py#L25

Python packages have a tendency to never get updated, so it might be a good idea to add a call to the github repo's raw yaml when loading the package. That way at least the prices stay up to date.

Something like this:

import requests
response = requests.get('https://raw.githubusercontent.com/AgentOps-AI/tokencost/main/tokencost/model_prices.yaml')
TOKEN_COSTS = yaml.safe_load(x.text)
areibman commented 10 months ago

Good idea.

There is a chance that we may not update the prices ASAP. As such, we should probably switch our price model to a repo that more actively keeps these prices updated. We grabbed our most recent prices from LiteLLM. To make things easier, I'm going to set them as the source of truth on pricing.

I've also included a fallback file model_prices.json in case the request doesn't go through. We'll keep this file regularly updated, though perhaps not as regular as LiteLLM.

I'm raising a pull request and major version update to include this #24

areibman commented 10 months ago

Fixed in latest release. Thanks for flagging @silver5753