Giskard-AI / giskard

🐢 Open-Source Evaluation & Testing for ML models & LLMs
https://docs.giskard.ai
Apache License 2.0
3.92k stars 250 forks source link

Generate synthetic questions with listing choices #1956

Open terry07 opened 3 months ago

terry07 commented 3 months ago

🚀 Feature Request

I suggest creating a new type of synthetic questions such that providing context and reference contexts, someone could ask "Which are the choices of that option?"

and get an answer as a list of cases.

🔈 Motivation

Isolating specific parts of text segments and answering consumes mainly the information from dedicated segments. When someone wants to combine more sources and make comparisons or list possible answers seems a more challenging use case.