JohnSnowLabs / langtest

Deliver safe & effective language models

http://langtest.org/

Apache License 2.0

490 stars 37 forks source link

Closed Prikshit7766 closed 8 months ago

Prikshit7766 commented 8 months ago

Description

llm_eval

we employ a more robust Language Model (LLM) to evaluate the model’s response. Test is passed if the score is higher than the configured min score.

supported tasks: question_answering

llm_eval:
    hub: openai
    min_score: 0.75
    model: gpt-3.5-turbo-instruct

min_score (float): Minimum score to pass the test.
model (string): LLM model use to evaluate the model reponse.
hub (string): Hub (library) for loading model from public models hub or from path

Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update