davidmigloz / langchain_dart

Build LLM-powered Dart/Flutter applications.
https://langchaindart.dev
MIT License
403 stars 69 forks source link

Add support for maxCompletionTokens and reasoningTokens in openai_dart #556

Closed davidmigloz closed 4 days ago

davidmigloz commented 4 days ago

maxTokens is deprecated in favor of maxCompletionTokens, and is not compatible with o1 series models.

Before:

final res = await client.createChatCompletion(
  request: CreateChatCompletionRequest(
    model: ChatCompletionModel.modelId('gpt-4o'),
    messages: [
      ChatCompletionMessage.system(
        content: 'You are a helpful assistant.',
      ),
      ChatCompletionMessage.user(
        content: ChatCompletionUserMessageContent.string('Hello!'),
      ),
    ],
    maxTokens: 2,
  ),
);

Now:

final res = await client.createChatCompletion(
  request: CreateChatCompletionRequest(
    model: ChatCompletionModel.modelId('o1-mini'),
    messages: [
      ChatCompletionMessage.system(
        content: 'You are a helpful assistant.',
      ),
      ChatCompletionMessage.user(
        content: ChatCompletionUserMessageContent.string('Hello!'),
      ),
    ],
    maxCompletionTokens: 2,
  ),
);

If you are using the o1 model, you can now see how many reasoning tokens it has consumed in the usage object:

final res = await client.createChatCompletion(...);
final reasoningTokens = res.usage?.completionTokensDetails?.reasoningTokens;