Chat CreateResponseUsage object is omitting some properties which have probably been have added in more modern versions of the API which are related to the tokens usage. I think cached_usage is specially important (two issues were already requesting this) as this would help users track if their prompts are being cached properly and could directly contribute to save money.
What:
Description:
Chat
CreateResponseUsage
object is omitting some properties which have probably been have added in more modern versions of the API which are related to the tokens usage. I thinkcached_usage
is specially important (two issues were already requesting this) as this would help users track if their prompts are being cached properly and could directly contribute to save money.Check the
usage
property for reference: https://platform.openai.com/docs/api-reference/chat/objectRelated:
Fixes #486 #492