Open msaelices opened 1 year ago
Test-ran the example cases using gpt-3.5-turbo
instead of the default gpt-4
(still waiting for the API). It is a good function to be added in.
I like the idea, the only thing is the implementation is very openai-specific, which is probably fine as long as we make it clear. How about we break down evals.py into evals/__init__.py
and evals/openai.py
.
Goal would be to import is_correct from promptimize.evals.openai
Should we begin with some sort of base eval function/class that could be used for other LLMs?
lain5etf7w@proton.me
Changes
is_correct()
evaluation function, which asks an LLM model to return if a response is correctProof-of-life