expectedparrot / edsl

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.
https://docs.expectedparrot.com
MIT License
176 stars 18 forks source link

Add describe() method for object info #630

Open rbyh opened 3 months ago

rbyh commented 3 months ago

Our objects could have a .describe() — makes an LLM call to get a description of what this object is about. Very useful in itself, but also we can embed and use in search immediately

ellipsis-dev[bot] commented 3 months ago

To have Ellipsis create a Pull Request with the implementation, comment below with "LGTM". If you want to make changes to the plan, comment below starting with "replan" with suggestions.


Implementation Steps

Step 1: Add describe() method to objects in 'data_transfer_models.py'

In the 'data_transfer_models.py' file located in the '/edsl' directory, add a new method called describe() to our objects. This method should make an LLM call to get a description of the object. Use the DummyModel class in /integration/test_langugage_model.py as a reference for making the LLM call. Ensure that the method is documented with docstrings to explain its purpose, parameters, and usage.

Step 2: Add tests for the describe() method in 'test_Base.py'

In the 'testBase.py' file located in the '/tests/base' directory, add a new test method for the describe() method. This test method should mock the LLM responses directly within the class and assert the expected outputs. Use the unittest framework, organize your tests into a class that inherits from unittest.TestCase, prefix your test method with `test` to denote it as a test case, and include a variety of test types in your test class to cover different aspects of the functionality of the describe() method.

Step 3: Update search functionality to use describe() method in 'edsl/Base.py'

In the 'edsl/Base.py' file, update the search functionality to use the new describe() method. Specifically, modify the search method in the PersistenceMixin class to incorporate the describe() method in the search operations. Ensure that the updated search functionality works as expected with the new describe() method and is consistent with the rest of the codebase.


Questions? Check out our documentation.

johnjosephhorton commented 3 months ago

They already have a "help" method - and tbh, repr and str are supposed to fill this role. Would you be happy if we just made sure the help method looked good for all objects?