feat: HuggingFaceAPIChatGenerator add token `usage` data

vblagoje commented 2 months ago

Why:

Adds token usage metadata to responses from HuggingFaceAPIChatGenerator. usage dictionary response meta field has the following two keys prompt_tokens and completion_tokens matching OpenAI format in token counting.

This feature, i.e. OpenAI token usage format compatibility, aside from chat generators interchangeability benefits, is needed for full support of Langfuse GENERATION token usage renderings in traces. See https://github.com/deepset-ai/haystack-private/issues/82 for more details.

What:

Added a new usage meta field with the keys prompt_tokens and completion_tokens to HuggingFaceAPIChatGenerator.
Modified the streaming and non-streaming response generation code to include the new usage information in the message metadata.

How can it be used:

# Get the usage information from the first reply
usage_info = response["replies"][0].meta["usage"]
prompt_tokens_used = usage_info["prompt_tokens"]
completion_tokens_used = usage_info["completion_tokens"]

print(f"Prompt tokens used: {prompt_tokens_used}, Completion tokens used: {completion_tokens_used}")

How did you test it:

Updated unit tests are included to ensure the presence of the usage meta field and its contained prompt_tokens and completion_tokens keys in the reply messages.
Tests cover non-streaming and streaming scenarios, ensuring compatibility with different API types.

Notes for the reviewer:

After this PR is merged ensure https://github.com/deepset-ai/haystack-private/issues/82 is closed as well
Support for HuggingFaceAPIGenerator hasn't been added intentionally as it is, as other non-chat generators, in the process of deprecation and eventual removal.

coveralls commented 2 months ago

Pull Request Test Coverage Report for Build 10957361582

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
11 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.09%) to 90.358%

Files with Coverage Reduction	New Missed Lines	%
components/classifiers/zero_shot_document_classifier.py	3	91.07%
utils/filters.py	3	96.91%
components/evaluators/llm_evaluator.py	5	95.08%
<!--	Total:	11	-->

Totals
Change from base Build 10899485113:	0.09%
Covered Lines:	7338
Relevant Lines:	8121

💛 - Coveralls

vblagoje commented 2 months ago

Please don't review this PR unless you are @anakin87

deepset-ai / haystack