princeton-nlp / SWE-bench

[ICLR 2024] SWE-bench: Can Language Models Resolve Real-world Github Issues?
https://www.swebench.com
MIT License
2k stars 348 forks source link

Image build error running gold_patch #250

Open jintheman opened 4 days ago

jintheman commented 4 days ago

Describe the bug

Running

python3 -m swebench.harness.run_evaluation \
    --dataset_name princeton-nlp/SWE-bench \
    --predictions_path gold \
    --max_workers 1 \
    --instance_ids pylint-dev__pylint-8898  \
    --run_id validate-gold \
;

Gives

Error building image pylint-dev__pylint-8898: 400 Client Error for http+docker://localhost/v1.46/containers/create?name=sweb.eval.pylint-dev__pylint-8898.validate-gold&platform=linux%2Fx86_64: Bad Request ("NanoCPUs can not be set, as your kernel does not support CPU CFS scheduler or the cgroup is not mounted")

This also happens to the following task instances

pylint-dev__pylint-6196 pylint-dev__pylint-6386 pylint-dev__pylint-6517 pylint-dev__pylint-6556 pylint-dev__pylint-6937 pylint-dev__pylint-7114 pylint-dev__pylint-7993 pylint-dev__pylint-8281 pylint-dev__pylint-8757 pylint-dev__pylint-8898

Steps/Code to Reproduce

git clone git@github.com:princeton-nlp/SWE-bench.git
cd SWE-bench
pip install -e .

python3 -m swebench.harness.run_evaluation \
    --dataset_name princeton-nlp/SWE-bench \
    --predictions_path gold \
    --max_workers 1 \
    --instance_ids pylint-dev__pylint-6196 pylint-dev__pylint-6386 pylint-dev__pylint-6517 pylint-dev__pylint-6556 pylint-dev__pylint-6937 pylint-dev__pylint-7114 pylint-dev__pylint-7993 pylint-dev__pylint-8281 pylint-dev__pylint-8757 pylint-dev__pylint-8898  \
    --run_id validate-gold \
;

Expected Results

Gold patches pass without error.

Actual Results

Error building image pylint-dev__pylint-8898: 400 Client Error for http+docker://localhost/v1.46/containers/create?name=sweb.eval.pylint-dev__pylint-8898.validate-gold&platform=linux%2Fx86_64: Bad Request ("NanoCPUs can not be set, as your kernel does not support CPU CFS scheduler or the cgroup is not mounted")

System Information

Ubuntu 22.04.4, Python 3.10, swebench 2.1.2.