GoogleCloudPlatform / genai-databases-retrieval-app

Apache License 2.0
212 stars 72 forks source link

chore: get prediction for eval dataset #414

Closed Yuan325 closed 4 months ago

Yuan325 commented 5 months ago

Add the function to get prediction for each of the queries from golden_dataset. Prediction is used as comparison to retrieve metrics.

Usage example:

from evaluation import run_llm_for_eval, goldens

# set up orchestration, session, set uuid
eval_list = await run_llm_for_eval(goldens, orchestration, session, session_id)