search

google / langfun

OO for LLMs
Apache License 2.0
122 stars 18 forks source link

Make `lf.eval.Evaluable.id` automatically inferred. #162

Closed copybara-service[bot] closed 5 months ago

copybara-service[bot] commented 5 months ago

Make lf.eval.Evaluable.id automatically inferred.

From prior experiments, we found that the id field is not very useful and complicates the eval framework's mental model. With this change, the ID of an leaf experiment will be automatically generated based on its class name and hash. As a result, users no longer need to specify id for both suites and evaluation tasks.

codecov-commenter commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.99%. Comparing base (e21eb33) to head (5e26cf6).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #162 +/- ## ========================================== - Coverage 96.01% 95.99% -0.02% ========================================== Files 55 55 Lines 4437 4445 +8 Branches 956 961 +5 ========================================== + Hits 4260 4267 +7 Misses 90 90 - Partials 87 88 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.