search

expectedparrot / edsl

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.
https://docs.expectedparrot.com
MIT License
96 stars 14 forks source link

Better tools for working with logprobs #260

Open johnjosephhorton opened 2 months ago

johnjosephhorton commented 2 months ago

results.select('kid_raw_model_response')[0]['raw_model_response.kid_raw_model_response'][0]['choices']

johnjosephhorton commented 2 months ago

Questions now have a "simple_ask" method that helps here. The problem is that our more advanced way of doing it kind of breaks things because the probabilities are over JSON formatting elements (blech)