search

princeton-nlp / SWE-bench

[ICLR 2024] SWE-bench: Can Language Models Resolve Real-world Github Issues?
https://www.swebench.com
MIT License
2k stars 348 forks source link

scikit-learn__scikit-learn images built error #218

Closed JiyangZhang closed 1 month ago

JiyangZhang commented 2 months ago

Describe the bug

Images for some scikit-learn instances fail to be built.

Looks like while installing dependencies, the package Cython conflicts with the installation.

Steps/Code to Reproduce

python -m swebench.harness.run_evaluation     --predictions_path gold     --max_workers 1     --instance_ids scikit-learn__scikit-learn-25500    --run_id test-sklearn

Expected Results

Evaluate with no errors

Actual Results

lib/python3.9/runpy.py:127: RuntimeWarning: 'swebench.harness.run_evaluation' found in sys.modules after import of package 'swebench.harness', but prior to execution of 'swebench.harness.run_evaluation'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Using gold predictions - ignoring predictions_path
Running 1 unevaluated instances...
Base image sweb.base.x86_64:latest already exists, skipping build.
Base images built successfully.
No environment images need to be built.
Running 1 instances...
  0%|                                                                                                                                            | 0/1 [00:00<?, ?it/s]Error building image sweb.eval.x86_64.scikit-learn__scikit-learn-25500:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1
Check (logs/build_images/instances/sweb.eval.x86_64.scikit-learn__scikit-learn-25500__latest/build_image.log) for more information.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:15<00:00, 15.04s/it]
All instances run.
Cleaning cached images...
Removed 0 images.
Total instances: 1
Instances submitted: 1
Instances completed: 0
Instances incomplete: 0
Instances resolved: 0
Instances unresolved: 0
Instances with empty patches: 0
Instances with errors: 1
Unstopped containers: 0
Unremoved images: 0
Report written to gold.test-sklearn.json

Logs:

2024-09-11 18:53:30,633 - INFO - return cythonize_one(*m)
2024-09-11 18:53:30,633 - INFO - File "/opt/miniconda3/envs/testbed/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
2024-09-11 18:53:30,633 - INFO - raise CompileError(None, pyx_file)
2024-09-11 18:53:30,633 - INFO - Cython.Compiler.Errors.CompileError: sklearn/linear_model/_cd_fast.pyx
2024-09-11 18:53:34,127 - INFO - multiprocessing.pool.RemoteTraceback:
  """
2024-09-11 18:53:34,127 - INFO - Traceback (most recent call last):
2024-09-11 18:53:34,127 - INFO - File "/opt/miniconda3/envs/testbed/lib/python3.9/multiprocessing/pool.py", line 125, in worker
2024-09-11 18:53:34,127 - INFO - result = (True, func(*args, **kwds))
2024-09-11 18:53:34,127 - INFO - File "/opt/miniconda3/envs/testbed/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
2024-09-11 18:53:34,127 - INFO - return list(map(*args))
2024-09-11 18:53:34,127 - INFO - File "/opt/miniconda3/envs/testbed/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1345, in cythonize_one_helper
2024-09-11 18:53:34,127 - INFO - return cythonize_one(*m)
2024-09-11 18:53:34,128 - INFO - File "/opt/miniconda3/envs/testbed/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
2024-09-11 18:53:34,128 - INFO - raise CompileError(None, pyx_file)
2024-09-11 18:53:34,128 - INFO - Cython.Compiler.Errors.CompileError: sklearn/utils/_random.pyx
2024-09-11 18:53:34,128 - INFO - """
2024-09-11 18:53:34,128 - INFO - 
2024-09-11 18:53:34,129 - INFO - The above exception was the direct cause of the following exception:
2024-09-11 18:53:34,129 - INFO - 
2024-09-11 18:53:34,129 - INFO - Traceback (most recent call last):
2024-09-11 18:53:34,129 - INFO - File "<string>", line 2, in <module>
2024-09-11 18:53:34,129 - INFO - File "<pip-setuptools-caller>", line 34, in <module>
2024-09-11 18:53:34,129 - INFO - File "/testbed/setup.py", line 681, in <module>
2024-09-11 18:53:34,130 - INFO - setup_package()
2024-09-11 18:53:34,130 - INFO - File "/testbed/setup.py", line 675, in setup_package
2024-09-11 18:53:34,130 - INFO - metadata["ext_modules"] = configure_extension_modules()
2024-09-11 18:53:34,130 - INFO - File "/testbed/setup.py", line 609, in configure_extension_modules
2024-09-11 18:53:34,130 - INFO - return cythonize_extensions(cython_exts)
2024-09-11 18:53:34,130 - INFO - File "/testbed/sklearn/_build_utils/__init__.py", line 98, in cythonize_extensions
2024-09-11 18:53:34,131 - INFO - return cythonize(
2024-09-11 18:53:34,131 - INFO - File "/opt/miniconda3/envs/testbed/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1145, in cythonize
2024-09-11 18:53:34,131 - INFO - result.get(99999)  # seconds
2024-09-11 18:53:34,131 - INFO - File "/opt/miniconda3/envs/testbed/lib/python3.9/multiprocessing/pool.py", line 771, in get
2024-09-11 18:53:34,131 - INFO - raise self._value
2024-09-11 18:53:34,131 - INFO - Cython.Compiler.Errors.CompileError: sklearn/utils/_random.pyx
2024-09-11 18:53:34,131 - INFO - [ 1/64] Cythonizing sklearn/__check_build/_check_build.pyx
2024-09-11 18:53:34,132 - INFO - [55/64] Cythonizing sklearn/utils/_random.pyx
2024-09-11 18:53:34,132 - INFO - [54/64] Cythonizing sklearn/utils/_openmp_helpers.pyx
2024-09-11 18:53:34,132 - INFO - [44/64] Cythonizing sklearn/svm/_newrand.pyx
2024-09-11 18:53:34,132 - INFO - [38/64] Cythonizing sklearn/neighbors/_partition_nodes.pyx
2024-09-11 18:53:34,132 - INFO - [51/64] Cythonizing sklearn/utils/_heap.pyx
2024-09-11 18:53:34,133 - INFO - [58/64] Cythonizing sklearn/utils/_sorting.pyx
2024-09-11 18:53:34,133 - INFO - [57/64] Cythonizing sklearn/utils/_seq_dataset.pyx
2024-09-11 18:53:34,133 - INFO - [59/64] Cythonizing sklearn/utils/_typedefs.pyx
2024-09-11 18:53:34,133 - INFO - [35/64] Cythonizing sklearn/metrics/cluster/_expected_mutual_info_fast.pyx
2024-09-11 18:53:34,133 - INFO - [22/64] Cythonizing sklearn/feature_extraction/_hashing_fast.pyx
2024-09-11 18:53:34,133 - INFO - [11/64] Cythonizing sklearn/decomposition/_cdnmf_fast.pyx
2024-09-11 18:53:34,133 - INFO - [18/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/common.pyx
2024-09-11 18:53:34,134 - INFO - [48/64] Cythonizing sklearn/tree/_utils.pyx
2024-09-11 18:53:34,134 - INFO - [15/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_bitset.pyx
2024-09-11 18:53:34,134 - INFO - [61/64] Cythonizing sklearn/utils/_weight_vector.pyx
2024-09-11 18:53:34,134 - INFO - [ 8/64] Cythonizing sklearn/cluster/_k_means_lloyd.pyx
2024-09-11 18:53:34,134 - INFO - [62/64] Cythonizing sklearn/utils/arrayfuncs.pyx
2024-09-11 18:53:34,135 - INFO - [13/64] Cythonizing sklearn/ensemble/_gradient_boosting.pyx
2024-09-11 18:53:34,135 - INFO - [34/64] Cythonizing sklearn/metrics/_pairwise_fast.pyx
2024-09-11 18:53:34,135 - INFO - [17/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_predictor.pyx
2024-09-11 18:53:34,135 - INFO - [26/64] Cythonizing sklearn/manifold/_barnes_hut_tsne.pyx
2024-09-11 18:53:34,135 - INFO - [52/64] Cythonizing sklearn/utils/_isfinite.pyx
2024-09-11 18:53:34,135 - INFO - [19/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/histogram.pyx
2024-09-11 18:53:34,137 - INFO - [43/64] Cythonizing sklearn/svm/_libsvm_sparse.pyx
  [21/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/utils.pyx
  [27/64] Cythonizing sklearn/manifold/_utils.pyx
  [63/64] Cythonizing sklearn/utils/murmurhash.pyx
  [60/64] Cythonizing sklearn/utils/_vector_sentinel.pyx
2024-09-11 18:53:34,137 - INFO - [24/64] Cythonizing sklearn/linear_model/_sag_fast.pyx
2024-09-11 18:53:34,137 - INFO - [ 4/64] Cythonizing sklearn/cluster/_dbscan_inner.pyx
2024-09-11 18:53:34,137 - INFO - [16/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_gradient_boosting.pyx
2024-09-11 18:53:34,137 - INFO - [20/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/splitting.pyx
2024-09-11 18:53:34,137 - INFO - [53/64] Cythonizing sklearn/utils/_logistic_sigmoid.pyx
2024-09-11 18:53:34,138 - INFO - [25/64] Cythonizing sklearn/linear_model/_sgd_fast.pyx
2024-09-11 18:53:34,138 - INFO - [50/64] Cythonizing sklearn/utils/_fast_dict.pyx
2024-09-11 18:53:34,138 - INFO - [ 5/64] Cythonizing sklearn/cluster/_hierarchical_fast.pyx
2024-09-11 18:53:34,138 - INFO - [46/64] Cythonizing sklearn/tree/_splitter.pyx
2024-09-11 18:53:34,138 - INFO - [30/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_base.pyx
2024-09-11 18:53:34,139 - INFO - [14/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_binning.pyx
2024-09-11 18:53:34,139 - INFO - [ 6/64] Cythonizing sklearn/cluster/_k_means_common.pyx
2024-09-11 18:53:34,139 - INFO - [33/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx
2024-09-11 18:53:34,139 - INFO - [45/64] Cythonizing sklearn/tree/_criterion.pyx
2024-09-11 18:53:34,139 - INFO - [29/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_argkmin.pyx
2024-09-11 18:53:34,140 - INFO - [ 7/64] Cythonizing sklearn/cluster/_k_means_elkan.pyx
2024-09-11 18:53:34,140 - INFO - [41/64] Cythonizing sklearn/svm/_liblinear.pyx
2024-09-11 18:53:34,140 - INFO - [31/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.pyx
2024-09-11 18:53:34,140 - INFO - [56/64] Cythonizing sklearn/utils/_readonly_array_wrapper.pyx
2024-09-11 18:53:34,140 - INFO - [ 9/64] Cythonizing sklearn/cluster/_k_means_minibatch.pyx
2024-09-11 18:53:34,141 - INFO - [40/64] Cythonizing sklearn/preprocessing/_csr_polynomial_expansion.pyx
2024-09-11 18:53:34,141 - INFO - [12/64] Cythonizing sklearn/decomposition/_online_lda_fast.pyx
2024-09-11 18:53:34,141 - INFO - [42/64] Cythonizing sklearn/svm/_libsvm.pyx
2024-09-11 18:53:34,141 - INFO - [ 2/64] Cythonizing sklearn/_isotonic.pyx
2024-09-11 18:53:34,141 - INFO - [39/64] Cythonizing sklearn/neighbors/_quad_tree.pyx
2024-09-11 18:53:34,142 - INFO - [32/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx
2024-09-11 18:53:34,142 - INFO - [49/64] Cythonizing sklearn/utils/_cython_blas.pyx
2024-09-11 18:53:34,142 - INFO - [64/64] Cythonizing sklearn/utils/sparsefuncs_fast.pyx
2024-09-11 18:53:34,142 - INFO - [10/64] Cythonizing sklearn/datasets/_svmlight_format_fast.pyx
2024-09-11 18:53:34,142 - INFO - [23/64] Cythonizing sklearn/linear_model/_cd_fast.pyx
2024-09-11 18:53:34,143 - INFO - [37/64] Cythonizing sklearn/neighbors/_kd_tree.pyx
2024-09-11 18:53:34,143 - INFO - [47/64] Cythonizing sklearn/tree/_tree.pyx
2024-09-11 18:53:34,143 - INFO - [36/64] Cythonizing sklearn/neighbors/_ball_tree.pyx
2024-09-11 18:53:34,143 - INFO - [28/64] Cythonizing sklearn/metrics/_dist_metrics.pyx
2024-09-11 18:53:34,143 - INFO - [ 3/64] Cythonizing sklearn/_loss/_loss.pyx
2024-09-11 18:53:34,394 - INFO - error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
2024-09-11 18:53:34,395 - INFO - full command: /opt/miniconda3/envs/testbed/bin/python -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/testbed/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /tmp/pip-pip-egg-info-265qlljb
2024-09-11 18:53:34,395 - INFO - cwd: /testbed/
2024-09-11 18:53:34,395 - INFO - Preparing metadata (setup.py): finished with status 'error'
2024-09-11 18:53:34,400 - INFO - error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
2024-09-11 18:53:34,945 - INFO - Removing intermediate container b419a96abda9
2024-09-11 18:53:34,946 - ERROR - Error: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1
2024-09-11 18:53:34,946 - ERROR - docker.errors.BuildError during sweb.eval.x86_64.scikit-learn__scikit-learn-25500:latest: The command '/bin/sh -c /bin/bash /root/setup_repo.sh' returned a non-zero code: 1

System Information

Python 3.9.19 swebench 2.0.13 Linux

HaomiaoPan commented 1 month ago

same problem

chehuier commented 1 month ago

same problem