GeneralBots / BotServer

LLM Orchestrator powered by langchain and Bot Framework V4 & several features including Whatsapp.
https://gb.pragmatismo.cloud
Other
64 stars 38 forks source link

Log how many tokens used in Langchain #405

Open rodrigorodriguez opened 8 months ago

rodrigorodriguez commented 8 months ago

import { ChatOpenAI } from 'langchain/chat_models/openai';

let totalCompletionTokens = 0; let totalPromptTokens = 0; let totalExecutionTokens = 0;

const llm = new ChatOpenAI({ callbacks: [ { handleLLMEnd: (output, runId, parentRunId?, tags?) => { const { completionTokens, promptTokens, totalTokens } = output.llmOutput?.tokenUsage; totalCompletionTokens += completionTokens ?? 0; totalPromptTokens += promptTokens ?? 0; totalExecutionTokens += totalTokens ?? 0; }, }, ], modelName: 'gpt-3.5-turbo-0613', });