Introduce `LMSamplingUsage` and make it a part of `LMSamplingResult`.

google / langfun

OO for LLMs

Apache License 2.0

122 stars 18 forks source link

Introduce `LMSamplingUsage` and make it a part of `LMSamplingResult`. #175

Closed copybara-service[bot] closed 4 months ago

copybara-service[bot] commented 4 months ago

Introduce LMSamplingUsage and make it a part of LMSamplingResult.

Previously we have Usage for OpenAI models, now we lift this up for all LLM use cases. This also add the usage object as the returning message's usage metadata.