Consider alternative handling of valid "None" responses to multiple choice questions

[rbyh]

It is fairly easy to create a multiple choice question where the model can reasonably want to not select a given option, but this question type throws an error if an option is not selected.

We should consider handling 'None' as we do with checkbox questions (allow it) or provide a clearer error message alerting the user to the issue.

Here I test valid "None" responses:

from edsl.questions import QuestionList, QuestionCheckBox, QuestionMultipleChoice, QuestionFreeText
from edsl import Scenario, Survey

q1 = QuestionList(
    question_name = "list",
    question_text = "List the countries mentioned in this text: {{ text }}"
)
q2 = QuestionCheckBox(
    question_name = "checkbox",
    question_text = "Select all the countries mentioned in this text: {{ text }}",
    question_options = ["United States", "Canada", "Mexico"]
)
q3 = QuestionMultipleChoice(
    question_name = "choice",
    question_text = "Select the first country mentioned in this text: {{ text }}",
    question_options = ["United States", "Canada", "Mexico"]
)
q4 = QuestionFreeText(
    question_name = "free",
    question_text = "Identify the countries mentioned in this text: {{ text }}"
)

texts = ["This is how you should brush your teeth.", "This is how to make coffee."]

scenarios = [Scenario({"text":t}) for t in texts]

survey = Survey([q1,q2,q3,q4])

The errors are only with QuestionMultipleChoice:

[johnjosephhorton]

Would it fix the problem to just modify the question answer validator so that None is a valid response to a MC question?

[rbyh]

Maybe? We don't want to get into validating all "None"-like responses, and then having to decide what we translate them to in results for consistency. We could try pairing it with an additional instruction to return "None" but I think that actually makes it less consistent with checkbox where we allow specified min/max, and I think consistency between these types is important.

I think my first preference is to leave it as is but add a clear error message that the model did not select a response for the multiple choice question. This has the benefit of alerting the user.

We could do the same for checkbox questions where the min/max have been specified but the model has not strayed.

I do think it's a situation where we do not necessarily want to mirror human surveys, where you can be prevented from moving on if you don't answer. It's arguably an improvement that we allow the model to implicitly point out that the question is bad, instead of forcing a non-sensible response to it.

[rbyh]

Could use optional parameter allow_none=True

[rbyh]