`lf.eval` framework improvements. - Githubissues

google / langfun

OO for LLMs

Apache License 2.0

100 stars 17 forks source link

`lf.eval` framework improvements. #191

Closed copybara-service[bot] closed 3 months ago

copybara-service[bot] commented 3 months ago

lf.eval framework improvements.

Introduce lf.eval.app_run to make it easy to build evaluation binaries.
Add token usage in experiment summary and result JSON.
Introduce lf.MappingError and capture LM response in error raised.
Print groundtruth and (final) answer on console for dryrun.