81 evaluation write function to run evaluation code during promptfoo evaluation - Githubissues

FullFact / health-misinfo-shared

Raphael health misinformation project, shared by Full Fact and Google

MIT License

0 stars 0 forks source link

81 evaluation write function to run evaluation code during promptfoo evaluation #94

Closed dearden closed 4 months ago

dearden commented 4 months ago

Fixes #81 .

Adapts the evaluation functions so they run with promptfoo

Still to do

Potentially use the actual functions from src/evaluation.py (or decide to use these as the default). Should be simple, but might require some tweaking of PYTHONPATH in the promptfoo call.
The expected format needs updating to reflect the current state of the prompt

Pull request checklist

[x] I have linked my PR to an issue
[x] I’ve used conventional commits
[ ] My branch is up-to-date with main
[ ] Where appropriate, I have added or updated tests
[ ] Where appropriate, I have updated documentation to reflect my changes