search

openai / openai-dotnet

The official .NET library for the OpenAI API
https://www.nuget.org/packages/OpenAI
MIT License
707 stars 60 forks source link

add tokenizer , #73

Open 3400442579 opened 1 week ago

3400442579 commented 1 week ago

var chatMessages= new ChatMessage[] { ChatMessage.CreateUserMessage(cont) }

int tokenCount=tokenizer (chatMessages); //

ChatCompletion completion = await _sdk.GetChatClient("gpt-4") .CompleteChatAsync( new ChatMessage[] { ChatMessage.CreateUserMessage(cont) } , new OpenAI.Chat.ChatCompletionOptions { MaxTokens = 8192 - tokenCount } );

StephenHodgson commented 2 days ago

Personally I think a better solution is to have an endpoint that tokenizes for you.

https://github.com/openai/openai-openapi/issues/205