search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
6.33k stars 4.23k forks source link

Tracking spent tokens #1217

Open developer992 opened 9 months ago

developer992 commented 9 months ago

How do we track spent tokens?

I've tracked down that this ChatCompletion.usage class holds this information but i am having hard time hunting all the calls down ... Is there an easier and more precise way?

pamelafox commented 9 months ago

Good question! I believe many customers specifically track token usage in either Azure API Management or Application Insights (Azure Monitor). Have you read through https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/monitoring to see what it recommends?