Closed chakravarthik27 closed 3 months ago
Harness setup
from langtest import Harness harness = Harness( task="question-answering", model={ "model": "llama3:8b", "hub": "ollama" }, data=[], config={ "evaluation": {"metric":"llm_eval","model":"gpt-4o","hub":"openai"}, "tests": { "defaults": { "min_pass_rate": 0.8, }, "safety": { "misuse": { "min_pass_rate": 0.8, "count": 10, } } } } )
generate, run, and get the report
harness.generate().run().report()
Harness setup
generate, run, and get the report