`lf.eval`: Fix a few issues - Githubissues

google / langfun

OO for LLMs

Apache License 2.0

100 stars 17 forks source link

`lf.eval`: Fix a few issues #200

Closed copybara-service[bot] closed 3 months ago

copybara-service[bot] commented 3 months ago

lf.eval: Fix a few issues

Dryrun to use the copy's examples to avoid mess up the current evaluation's state.
Reset examples when starting to run.
Fix usage aggregation logic when the LLM does not provide usage information.

This avoids the state of the current evaluation get messed up if calling self.examples has a side effect.