search

expectedparrot / edsl

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.
https://docs.expectedparrot.com
MIT License
192 stars 19 forks source link

Failing notebooks with new __repr__ approach #1302

Open johnjosephhorton opened 1 day ago

johnjosephhorton commented 1 day ago 
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/adding_metadata.ipynb] PASSED                                                                     [  1%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/agentifying_responses.ipynb] FAILED                                                               [  3%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/analyze_customer_call.ipynb] PASSED                                                               [  5%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/analyze_evaluations.ipynb] FAILED                                                                 [  7%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/analyzing_interviews.ipynb] PASSED                                                                [  9%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/batching_results.ipynb] FAILED                                                                    [ 11%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/ces_data_edsl.ipynb] PASSED                                                                       [ 13%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/comparing_model_responses.ipynb] PASSED                                                           [ 15%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/concept_induction.ipynb] PASSED                                                                   [ 17%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/conduct_interview.ipynb] PASSED                                                                   [ 19%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/data_cleaning.ipynb] PASSED                                                                       [ 21%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/data_labeling_agent.ipynb] PASSED                                                                 [ 23%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/data_labeling_example.ipynb] PASSED                                                               [ 25%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/demo_notebook.ipynb] PASSED                                                                       [ 27%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/digital_twin.ipynb] FAILED                                                                        [ 29%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/docs_questions.ipynb] PASSED                                                                      [ 31%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/edsl_intro.ipynb] PASSED                                                                          [ 33%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/edsl_polling.ipynb] PASSED                                                                        [ 35%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/edsl_with_cloud_providers.ipynb] FAILED                                                           [ 37%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/edsl_with_offline_inference_services.ipynb] FAILED                                                [ 39%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/estimating_costs.ipynb] FAILED                                                                    [ 41%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/evaluating_job_posts.ipynb] PASSED                                                                [ 43%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/example_agent_dynamic_traits.ipynb] PASSED                                                        [ 45%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/explore_llm_biases.ipynb] PASSED                                                                  [ 47%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/explore_survey_contexts.ipynb] PASSED                                                             [ 49%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/features_checklist.ipynb] PASSED                                                                  [ 50%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/filestore_examples.ipynb] PASSED                                                                  [ 52%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/free_responses.ipynb] PASSED                                                                      [ 54%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/google_form_to_edsl.ipynb] PASSED                                                                 [ 56%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/grading_experiment.ipynb] FAILED                                                                  [ 58%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/image_scenario_example.ipynb] PASSED                                                              [ 60%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/import_agents.ipynb] PASSED                                                                       [ 62%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/next_token_probs.ipynb] PASSED                                                                    [ 64%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/nps_survey.ipynb] PASSED                                                                          [ 66%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/qualitative_research.ipynb] PASSED                                                                [ 68%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/question_extract_example.ipynb] PASSED                                                            [ 70%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/question_loop_scenarios.ipynb] PASSED                                                             [ 72%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/random_numbers.ipynb] PASSED                                                                      [ 74%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/research_methods.ipynb] PASSED                                                                    [ 76%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/save_load_objects_locally.ipynb] PASSED                                                           [ 78%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/scenario_from_pdf.ipynb] PASSED                                                                   [ 80%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/scenario_list_wikipedia.ipynb] PASSED                                                             [ 82%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/scenariolist_unpivot.ipynb] PASSED                                                                [ 84%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/scenarios_filestore_example.ipynb] PASSED                                                         [ 86%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/skip_logic_scenarios.ipynb] PASSED                                                                [ 88%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/starter_tutorial.ipynb]
johnjosephhorton commented 1 day ago

Looking good on these: 

egration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/summarizing_transcripts.ipynb] PASSED                                                             [ 92%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/testing_training_data.ipynb] PASSED                                                               [ 94%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/updating_agents.ipynb] PASSED                                                                     [ 96%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/writing_style.ipynb] PASSED                                                                       [ 98%]
integration/active/test_notebooks.py::test_notebook_execution[docs/notebooks/yoga_studio_name_survey.ipynb]
johnjosephhorton commented 1 day ago

Failure reports: https://www.dropbox.com/scl/fi/04tx0f6zq3gj6jne86gvs/failures.rtf?rlkey=g0q15febx9uteios1b6bxdnqf&dl=0