search

cohere-ai / cohere-python

Python Library for Accessing the Cohere API
https://docs.cohere.ai
MIT License
278 stars 60 forks source link

When using BedrockClient, the response does not contain meta fields for billed tokens #531

Closed c64er4ever closed 3 weeks ago

c64er4ever commented 1 month ago

Hi, I am using both Client and BedrockClient. When using Client, I can access fields: response.meta.billed_units.input_tokens response.meta.billed_units.output_tokens However, when using BedrockClient, the response does not seem to contain any meta fields, so I cannot check the number of billed input and output tokens. I tested direct calls to Bedrock via boto3 invoke, and in this case I saw that I am getting the billed tokens using special x-amzn-bedrock header fields. It seems that currently the client does not handle these fields. Thanks Guy

billytrend-cohere commented 1 month ago

Hey @c64er4ever ! Thanks for letting us know about this. I will have a fix for you this week or next. Best, Billy

c64er4ever commented 1 month ago

That's great, thanks Billy!

billytrend-cohere commented 1 month ago

I have a PR here @c64er4ever, will ping when it's released https://github.com/cohere-ai/cohere-python/pull/537

billytrend-cohere commented 3 weeks ago

This is released in the latest python version 5.6.2