search

mozilla-ai / lm-buddy

Your buddy in the (L)LM space.
Apache License 2.0
64 stars 3 forks source link

RD2024-147: Implement LMBuddy class as generic job runner #88

Closed sfriedowitz closed 6 months ago

sfriedowitz commented 6 months ago

What's changing

Combination of the two PRs:

1) https://github.com/mozilla-ai/lm-buddy/pull/84 2) https://github.com/mozilla-ai/lm-buddy/pull/87

Additional notes for reviewers

Note that I have made a few breaking changes to the CLI and the LMHarnessJobConfig data structure here. Also for @imtihan , the general entrypoint structure is the same for your Ragas work, but now returns an EvaluationResult return type that should be accounted for.