expectedparrot / edsl

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.
https://docs.expectedparrot.com
MIT License
187 stars 19 forks source link

Question `loop()` method allows non-unique question names #1172

Open rbyh opened 2 weeks ago

rbyh commented 2 weeks ago

Example:

Eg, if I use a scenario list with multiple keys and only pass one of them to the question name this will generate different questions with the same names:

from edsl import QuestionYesNo, ScenarioList, Scenario

q = QuestionYesNo(
    question_name = "enjoy_{{ sport }}",
    question_text = "Do you enjoy {{ sport }} and {{ non_sport }}?"
)

s = ScenarioList([
    Scenario({"sport":s, "non_sport":n}) for s in ["fishing", "golf"] for n in ["reading", "painting"]
])

questions = q.loop(s)
questions

Returns:

[Question('yes_no', question_name = """enjoy_fishing""", question_text = """Do you enjoy fishing and reading?""", question_options = ['No', 'Yes']),
 Question('yes_no', question_name = """enjoy_fishing""", question_text = """Do you enjoy fishing and painting?""", question_options = ['No', 'Yes']),
 Question('yes_no', question_name = """enjoy_golf""", question_text = """Do you enjoy golf and reading?""", question_options = ['No', 'Yes']),
 Question('yes_no', question_name = """enjoy_golf""", question_text = """Do you enjoy golf and painting?""", question_options = ['No', 'Yes'])]

Because the user here is trying to set the question names (as opposed to accepting the incremented default question names) I think there should be a warning message so they can decide how to modify them.

johnjosephhorton commented 1 week ago

I don't think this is a bug - it's just constructing how the user specified. A list of question objects is not necessarily a 'survey' so I'm reluctant to enforce uniqueness at this point before we know what the user wants to do.T They could do

s = Survey([questions[0], questions[2]]) 

and then this is fine.

rbyh commented 1 week ago

Right and then they'll get an error that question names are not unique. It just seems to me that if you're using the loop method you more than likely intend to use all the questions in the same survey, so why make you re-do it in a way that might feel fussy.

johnjosephhorton commented 1 week ago

You will not get an error with my example because they are unique - question[0] and question[2] have distinct names. I'd rather do something like add an argument to "loop" that's something like "force_unique_names" or something but not take secret actions on user's behalf. When I create a template, I'd expect it to be followed exactly, not changed secretly w/o notice.

rbyh commented 1 week ago

Ok that makes sense. Let's add optional parameters for loop: