Evaluation test for checking if the quote appears in the chunk - Githubissues

FullFact / health-misinfo-shared

Raphael health misinformation project, shared by Full Fact and Google

MIT License

0 stars 0 forks source link

Evaluation test for checking if the quote appears in the chunk #99

Closed dearden closed 1 month ago

dearden commented 2 months ago

Fixes #80 .

We want to check if the quotes pulled out by the model actually appear directly in the chunk text given to the LLM as input.

This test will check that (and will be run by promptfoo)

Pull request checklist

[x] I have linked my PR to an issue
[x] I’ve used conventional commits
[x] My branch is up-to-date with main
[ ] Where appropriate, I have added or updated tests
[ ] Where appropriate, I have updated documentation to reflect my changes