expectedparrot / edsl

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.
https://docs.expectedparrot.com
MIT License
168 stars 18 forks source link

Results `filter()` method requires component prefix and throws misleading exception ("No data to select from---the Results object is empty.") if not present; inconsistent with related methods #795

Closed rbyh closed 1 month ago

rbyh commented 1 month ago

The filter() method currently requires the component prefix to be included, throwing an exception that the filter yields no results, while related methods select() and sort_by() do not require prefixes. This is likely confusing and unexpected to a user. Compare:

from edsl import QuestionFreeText

q = QuestionFreeText(
    question_name = "q1",
    question_text = "How do you feel today?"
)

results = q.run()

(results
 .sort_by("model")
 .filter("model.model == 'gpt-4-1106-preview'")
 .select("model")
 .print(format="rich")
)

This yields:

┏━━━━━━━━━━━━━━━━━━━━┓
┃ model              ┃
┃ .model             ┃
┡━━━━━━━━━━━━━━━━━━━━┩
│ gpt-4-1106-preview │
└────────────────────┘

vs

(results
 .sort_by("model")
 .filter("model.model == 'gpt-4-1106-preview'")
 .select("model")
 .print(format="rich")
)

This throws an exception (no results based on the filter):

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[20], line 4
      1 (results
      2  .sort_by("model")
      3  .filter("model == 'gpt-4-1106-preview'")
----> 4  .select("model")
      5  .print(format="rich")
      6 )

File [~/edsl/edsl/results/Results.py:747](http://localhost:8888/lab/tree/~/edsl/edsl/results/Results.py#line=746), in Results.select(self, *columns)
    734 """
    735 Select data from the results and format it.
    736 
   (...)
    743 Dataset([{'answer.how_feeling': ['OK', 'Great', 'Terrible', 'OK']}])
    744 """
    746 if len(self) == 0:
--> 747     raise Exception("No data to select from---the Results object is empty.")
    749 if not columns or columns == ("*",) or columns == (None,):
    750     columns = ("*.*",)

Exception: No data to select from---the Results object is empty.
johnjosephhorton commented 1 month ago

It's only "model" that does this - i now throw an exception for a 'bare' reference to model.

rbyh commented 1 month ago

I'll mention in docs