Error when running evaluation script - ModuleNotFoundError: No module named 'swebench'

[brombaut]

When I try to run the run_evaluation.sh script (or the python run_evaluation.py --all --the --args --... script for that matter) from the swebench/harness directory, I get the following error:

Traceback (most recent call last):
  File "/swebench_workspace/SWE-bench/swebench/harness/run_evaluation.py", line 15, in <module>
    from swebench.harness.constants import (
ModuleNotFoundError: No module named 'swebench'

Adding the SWE-Bench directory to my PYTHONPATH like below before running the script solves the issue

PYTHONPATH=/swebench_workspace/SWE-bench:$PYTHONPATH  ./run_evaluation.sh

[john-b-yang]

Hi @brombaut thanks for pointing this out, some of this functionality likely changed since we restructured the repository to be reflective of how the swebench pypi is built.

Thank you for putting down the tip. It is also possible to run evaluation using the swebench pypi package, which doesn't require building from source. You can find an example here.

[psykhi]

Hi!

Just cloned the repo with a fresh conda install as well, same thing happened to me. Should this issue really be closed?

[brombaut]

Yeah I pulled the updated changes, fresh environment setup, still got this error. I will say that doing pip install swebench did fix the issue, but I want to make local changes, so that doesn't really work for me (and also the same i mentioned in the original issue report still works, so that's the workaround)

[belindamo]

I'm also getting this error and I cloned and set up this repo today. Though for me the error happened when I ran ./collect/run_get_tasks_pipeline.sh as part of the Collections tutorial