expectedparrot / edsl

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.
https://docs.expectedparrot.com
MIT License
97 stars 14 forks source link

Create prompts that can use QuestionFreeText (or other question types) to return a dictionary #115

Open rbyh opened 4 months ago

rbyh commented 4 months ago

It can be useful to have the agent return a dictionary (eg agent traits) but this is not addressed with existing question type prompts. Eg trying to manipulate QuestionFreeText to do this throws an error because the response is not a string:

q_ranges = QuestionFreeText(
    question_name = "ranges",
    question_text = """You are being asked to design a survey about job posts at Upwork.
    Provide a dictionary where the keys are dimensions of freelancers
    and the values are lists of possible options for those dimensions."""
)
q_ranges.by(model).run().select("ranges").print()
image
rbyh commented 4 months ago

Using QuestionList to return a list of dicts is workable option:

image
rbyh commented 4 months ago

"Return the dict as a single item in a list" is hacky use of QuestionList that works:

image
johnjosephhorton commented 4 months ago

Another approach is to do something like this where you break in into steps:

https://embed.deepnote.com/dceaad48-76c8-4ecb-a682-3eea29a1bfb2/3ca7ff714cdb4e978b93345638d23d2c/7d72edc51bff40edab51344cf0c6ad24?height=1797.0999755859375