langchain-core 0.2.2 released a standard field to store usage metadata returned from chat model responses, such as input / output token counts. AIMessage objects have a .usage_metadata attribute which can hold a UsageMetadata dict. For now it is only holding token counts. Standardizing this information makes it simpler to track in monitoring / observability platforms and similar applications.
Here we unpack usage metadata returned by GenAI and Vertex gemini and non-gemini models onto AIMessages generated by chat models.
langchain-core 0.2.2 released a standard field to store usage metadata returned from chat model responses, such as input / output token counts. AIMessage objects have a
.usage_metadataattribute which can hold a UsageMetadata dict. For now it is only holding token counts. Standardizing this information makes it simpler to track in monitoring / observability platforms and similar applications.Here we unpack usage metadata returned by GenAI and Vertex gemini and non-gemini models onto AIMessages generated by chat models.