Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.
Currently, we use an encoding scheme to present options:
0: Option A
1: Option B
2: Option C
This was partially to make it easier to parse responses---we don't need to have the response match some long exact string. However, these keys likely affect responses in some cases (0 = False, 1 = True).
Proposal
We might consider letting the user control the scheme or possibly doing away with it and asking for them model to return the literal option.
rbyh
commented
5 months ago
Currently, we use an encoding scheme to present options:
0: Option A 1: Option B 2: Option C
This was partially to make it easier to parse responses---we don't need to have the response match some long exact string. However, these keys likely affect responses in some cases (0 = False, 1 = True).
Proposal
We might consider letting the user control the scheme or possibly doing away with it and asking for them model to return the literal option.