Bugs in run_setup

[tangken333]

Hii, I am using the run_setup.py to build the swe-bench environment. But I face two problems there: My command is: python harness/run_setup.py --log_dir logs --testbed testbed --result_dir setup_result --subset_file /opt/PatchingAgent/swe_lite_tasks.txt --num_processes 60 My environment is in the docker with the conda of swe-bench

1 it does not stop build just stop by here for hours.

2 got the errors:

2024-09-17 20:12:14,043 - ERROR - Error stderr:                                                                                                                                                                                                                                                                         
[Errno 2] No such file or directory: '/root/miniconda3/pkgs/packaging-24.1-py39h06a4308_0.conda.partial' -> '/root/miniconda3/pkgs/packaging-24.1-py39h06a4308_0.conda'                                                                                                                                                 
[Errno 2] No such file or directory: '/root/miniconda3/pkgs/packaging-24.1-py39h06a4308_0.conda.partial' -> '/root/miniconda3/pkgs/packaging-24.1-py39h06a4308_0.conda'                                                                                                                                                 
[Errno 2] No such file or directory: '/root/miniconda3/pkgs/exceptiongroup-1.2.0-py39h06a4308_0.conda.partial' -> '/root/miniconda3/pkgs/exceptiongroup-1.2.0-py39h06a4308_0.conda'                                                                                                                                     
[Errno 2] No such file or directory: '/root/miniconda3/pkgs/pytest-7.4.4-py39h06a4308_0.conda.partial' -> '/root/miniconda3/pkgs/pytest-7.4.4-py39h06a4308_0.conda'                                                                                                                                                     
[Errno 2] No such file or directory: '/root/miniconda3/pkgs/python-3.9.19-h955ad1f_1.conda'                                                                                                                                                                                                                             

2024-09-17 20:12:14,045 - ERROR - Error traceback: Traceback (most recent call last):
  File "/opt/SWE-bench/harness/context_manager.py", line 49, in __call__
    output = subprocess.run(cmd, **combined_args)
  File "/root/miniconda3/envs/swe-bench/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/miniconda3/bin/conda', 'create', '-n', 'setup_psf__requests__2.3', 'python=3.9', 'pytest', '-y']' returned non-zero exit status 1.

[yuntongzhang]

Thank you for the report.

For 1: The issue seems to be you are using too many parallel processes. Conda is not thread-safe, so using a large num_processes may result in deadlock. You can see more about setting up here. The safest is to not use parallelization at all; however, you can try with a smaller num_processes first (e.g. 8) to speed it up, and then switch to no parallelization if that also results in deadlock.

For 2: Does the error appear if you run setup on that bug alone? I will look into it more and try to reproduce the error.