gamification-survey-platform / client

0 stars 0 forks source link

genAI feedback concerns incorrect question #55

Open drwlodarski opened 5 months ago

drwlodarski commented 5 months ago

The feedback quality evaluation should be applied to two questions from "Open-ended section":

However it seems to apply to the first 2 open-ended questions that appear in the survey:

Screenshot 2024-04-14 at 10 48 46

How is it determined which questions should genAI provide feedback on?

EDIT: it seems that it applies to all open-ended questions. How is the rubric score derived then? For all 3 open-ended answers? This probably have impact on the open-ended answers quality badges.

audrey-zhou commented 5 months ago

Oh currently the feedback would provide for all text area questions - I thought only the open-ended questions would be text area. Then shall I just hardcode the open-ended question text to the logic for providing gpt feedback? Or we can add another Boolean field is_gpt_applicable to the question?

drwlodarski commented 5 months ago

For the sake of the experiment I'll remove the first open-ended question - it's not that important. Moving forward, we should probably add a configuration button for open-ended questions during the survey creation process. Let's discuss this during Tuesday's meeting. Something to consider: impact on calculating the quality rubric score (when we have more than 2 open-ended questions).