openai / evals

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Other
14.35k stars 2.54k forks source link

When installing the project dependencies, i got: "ERROR: Could not build wheels for greenlet, which is required to install pyproject.toml-based projects" #1513

Closed JuanmaMenendez closed 3 months ago

JuanmaMenendez commented 3 months ago

Describe the bug

After cloning the repo and running: pip install . I get: "ERROR: Could not build wheels for greenlet, which is required to install pyproject.toml-based projects"

I am pretty sure this is related to the fixed version dependency of playwright==1.32.1, and some kind of version conflict with greenlet or another dependency.

dependencies = [ ... "playwright==1.32.1", ... ]

Because if I remove the exact version number (1.32.1), the installation works as expected. I am not sure if there are some consequences of not installing that specific version.

To Reproduce

1- clone the repo and create an environment 2- run pip install . 3- the installation will fail and you will see: ERROR: Could not build wheels for greenlet, which is required to install pyproject.toml-based projects

Code snippets

No response

OS

macOs 13.6.1

Python version

v3.9.6

Library version

2.0.0.post1

JuanmaMenendez commented 3 months ago

Hi @danesherbs, I noticed you added "playwright==1.32.1", do you have any comments about having a fixed version for that library? Thanks

shyam-krish commented 3 months ago

I ran into the same issue with macOS 14.1.1 and Python version v3.9.6. Removing the fixed version for the playwright library fixed it for me as well. Appreciate it @JuanmaMenendez!

Worth providing some guidance here to save others hours of debugging time 😄

danesherbs commented 3 months ago

Thanks for raising this @JuanmaMenendez!

For context, the playwright module is needed to run the Multi-Step Web Tasks eval, which uses Web Arena environments, which require playwright==1.32.1. However, I've verified that we can relax the version constraint on playwright and can still successfully run the Multi-Step Web Task test task using oaieval dummy multistep-web-tasks.

I've opened a PR updating pyproject.toml to fix the issue :)