search

tghamm / Anthropic.SDK

An unofficial C#/.NET SDK for accessing the Anthropic Claude API
https://www.nuget.org/packages/Anthropic.SDK
MIT License
55 stars 10 forks source link

How to calculate usage? #9

Closed dmki closed 1 year ago

dmki commented 1 year ago

Hi,

In GPT SDKs we have 2 properties of response - Input tokens and output tokens that were used. If I understand right, here we are getting only strings of Claude's response, so how would you suggest to calculate the cost of the call?

Thanks!

tghamm commented 1 year ago

This actually sent me down quite the rabbit hole. They haven't really documented how their tokens are calculated in depth, and nothing is returned from the API, you have to dive into their typescript/python sdks, where they are using Huggingface's rust/python tokenizer library with their own custom BPE model (aka different from OpenAI's models). The short version is, I was able to extract from that model, the merges and the vocab files, and use ML.NET to import them and calculate the tokens used in a string. It'll add a few dependencies, which I need to sort through, but TLDR, I think I can work out an addition that lets you accurately calculate tokens. I'll get back to you in a couple of days when I've got something stable and tested.

tghamm commented 1 year ago

Version 1.2 has been released, which includes an extension method for counting tokens accurately. Please see the readme or the tests for example usages! Thanks for raising the issue.