Open klieret opened 1 month ago
I'm honestly confused how the dependency ends up there.
It appears directly in setup_env.sh
#!/bin/bash
set -euxo pipefail
source /opt/miniconda3/bin/activate
conda create -n testbed python=3.9 -y
cat <<'EOF_59812759871' > $HOME/requirements.txt
flake8
flake8-docstrings
flake8-black
doc8
Pygments
coverage
hypothesis
pytest
pytest-cov
pytest-sugar
mypy
types-toml
types-pkg_resources
types-chardet
requests
EOF_59812759871
conda activate testbed && python -m pip install -r $HOME/requirements.txt
rm $HOME/requirements.txt
conda activate testbed
so I would assume I should either find it in our constants.py
, or somewhere in the requirements.txt
. But the former is not the case, and the latter also doesn't seem to be (e.g., here's requirements.txt
for v0.7)...
I'm honestly confused how the dependency ends up there.
@klieret I think it's redirected to "requirement_dev.txt"
by get_requirements()
-> get_requirements_by_commit()
Like MAP_REPO_VERSION_TO_SPECS
says the install package type is "requirements.txt"
, then get_requirements()
is called; it looks in MAP_REPO_TO_REQS_PATHS
and got "requirement_dev.txt"
>>> from swebench.harness.constants import MAP_REPO_VERSION_TO_SPECS, MAP_REPO_TO_REQS_PATHS
2024-08-11 18:12:17,556 - datasets - INFO - PyTorch version 2.3.0 available.
>>> MAP_REPO_VERSION_TO_SPECS['sqlfluff/sqlfluff']['0.6']
{'python': '3.9', 'packages': 'requirements.txt', 'install': 'python -m pip install -e .', 'test_cmd': 'pytest -rA'}
>>> MAP_REPO_TO_REQS_PATHS['sqlfluff/sqlfluff']
['requirements_dev.txt']
Thanks @HejiaZ2023, indeed https://github.com/sqlfluff/sqlfluff/blob/0.7.0/requirements_dev.txt shows types-pkg_resources
in there.
In fact, they only removed it last week when the package was yanked: https://github.com/sqlfluff/sqlfluff/pull/6039
@klieret any idea when this will be fixed? is there a workaround until then?
I'll open a PR with a workaround later today/early tomorrow (there's a few ways to do this and Carlos/John also wanted to take a look, so I was waiting for feedback)
@klieret / @john-b-yang / @carlosejimenez - I wanted to circle back on this issue and see if you got a chance to look at it. Thanks
See also https://github.com/princeton-nlp/SWE-agent/issues/707, first reported by @waterson samizdis.
The package
types-pkg-resources
package was yanked from pypi (see pypi.org/project/types-pkg-resources/#history and pypi.org/project/types-pkg-resources).This means that any package with
requirements.txt
that specifies it will fail installing.Steps to reproduce:
all_preds.jsonl
forsqlfluff__sqlfluff-1625
, like the one belowGrep for
yanked
to find error inbuild_image.log
.Possible fix: Identify all versions of
sqlfluff
that havetypes-pkg-resources
in therequirements.txt
and explicitly pin and installtypes-pkg-resources
(probably to the latest version) in the extra pip packages.