Closed ivan4722 closed 1 week ago
Issue running SWE-bench evaluation, potential conda or OS error.
Trying to run eval with this in the ./run_evaluation.sh script:
#!/bin/bash python run_evaluation.py \ --predictions_path "../SWE-agent/trajectories/ixiong/gpt-3.5-turbo-0125__SWE-bench_Lite__default__t-0.00__p-0.95__c-2.00__install-1/all_preds.jsonl" \ --swe_bench_tasks "../../../swe-bench-lite.json" \ --log_dir "./log_dir" \ --testbed "./testbed" \ --skip_existing \ --timeout 900 \ --verbose
My conda environment is initialized. I think this may be because im on MAC OSX not linux, but I do know the fix. I tried adding this:
"arch_specific_packages": { "aarch64": "gxx_linux-aarch64 gcc_linux-aarch64 make", "osx-64": "gxx_osx-64 gcc_osx-64 make", },
to /swebench/harness/constants.py, but it did not work.
SWE-bench evaluation works and results is outputted
2024-06-24 11:55:25,843 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__3.4; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/3.4/tmp1i7l7v2n/miniconda3/bin/activate sphinx-doc__sphinx__3.4 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:29,787 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__3.3; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/3.3/tmpevyth59r/miniconda3/bin/activate sphinx-doc__sphinx__3.3 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:29,788 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__4.0; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/4.0/tmpq1_lk_y2/miniconda3/bin/activate sphinx-doc__sphinx__4.0 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:30,774 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__3.1; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/3.1/tmplycm1z2o/miniconda3/bin/activate sphinx-doc__sphinx__3.1 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:30,972 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__3.5; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/3.5/tmpchkpk5tt/miniconda3/bin/activate sphinx-doc__sphinx__3.5 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:31,084 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__5.1; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/5.1/tmpbiflqh6g/miniconda3/bin/activate sphinx-doc__sphinx__5.1 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:31,098 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__3.2; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/3.2/tmpg4k_tfp9/miniconda3/bin/activate sphinx-doc__sphinx__3.2 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:31,645 - testbed - INFO - Installing arch-specific packages for sphinx-doc__sphinx__7.1; Command: . /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/7.1/tmph1sibeup/miniconda3/bin/activate sphinx-doc__sphinx__7.1 && conda install gxx_linux-64 gcc_linux-64 make -y 2024-06-24 11:55:35,035 - testbed - ERROR - Error: Command '. /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/3.4/tmp1i7l7v2n/miniconda3/bin/activate sphinx-doc__sphinx__3.4 && conda install gxx_linux-64 gcc_linux-64 make -y' returned non-zero exit status 1. 2024-06-24 11:55:35,056 - testbed - ERROR - Error stdout: Channels: - defaults Platform: osx-64 Collecting package metadata (repodata.json): ...working... done Solving environment: ...working... failed PackagesNotFoundError: The following packages are not available from current channels: - gcc_linux-64 - gxx_linux-64 Current channels: - defaults To search for alternate channels that may provide the conda package you're looking for, navigate to https://anaconda.org and use the search bar at the top of the page. 2024-06-24 11:55:35,087 - testbed - ERROR - Error traceback: Traceback (most recent call last): File "/Users/ixiong/miniconda3/envs/swe-bench/lib/python3.9/site-packages/swebench/harness/context_manager.py", line 82, in __call__ output = subprocess.run(cmd, **combined_args) File "/Users/ixiong/miniconda3/envs/swe-bench/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '. /Users/ixiong/Desktop/csvtransfer/SWE-bench/swebench/harness/testbed/d2d0ffafc9/sphinx/3.4/tmp1i7l7v2n/miniconda3/bin/activate sphinx-doc__sphinx__3.4 && conda install gxx_linux-64 gcc_linux-64 make -y' returned non-zero exit status 1.
Python 3.9.19 swebench 1.1.5 MAC OS 14.5 M3
Describe the bug
Issue running SWE-bench evaluation, potential conda or OS error.
Steps/Code to Reproduce
Trying to run eval with this in the ./run_evaluation.sh script:
My conda environment is initialized. I think this may be because im on MAC OSX not linux, but I do know the fix. I tried adding this:
to /swebench/harness/constants.py, but it did not work.
Expected Results
SWE-bench evaluation works and results is outputted
Actual Results
System Information
Python 3.9.19 swebench 1.1.5 MAC OS 14.5 M3