search

nuprl / MultiPL-E

A multi-programming language benchmark for LLMs
https://nuprl.github.io/MultiPL-E/
Other
200 stars 38 forks source link

error in execution #159

Closed BingyuanZhang closed 2 days ago

BingyuanZhang commented 2 days ago

When I execute using the non-container method, that is, running the commands "cd evaluation/src" and "python3 main.py --dir ../../tutorial --output-dir ../../tutorial --recursive", I find an error reported.

Traceback (most recent call last): File "/mmu_nlp_ssd/zhangbingyuan/benchmark/MultiPL-E/evaluation/src/main.py", line 192, in main() File "/mmu_nlp_ssd/zhangbingyuan/benchmark/MultiPL-E/evaluation/src/main.py", line 155, in main evaluate_problem(args.output_dir, file, args.max_workers, args.dir) File "/mmu_nlp_ssd/zhangbingyuan/benchmark/MultiPL-E/evaluation/src/main.py", line 97, in evaluate_problem for j in executor.map(lambda index: cached_eval_script(problem, index), range(min_problem, num_problems)): File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator yield _result_or_cancel(fs.pop()) File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel return fut.result(timeout) File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.get_result() File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/concurrent/futures/_base.py", line 403, in get_result raise self._exception File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) File "/mmu_nlp_ssd/zhangbingyuan/benchmark/MultiPL-E/evaluation/src/main.py", line 97, in for j in executor.map(lambda index: cached_eval_script(problem, index), range(min_problem, num_problems)): File "/mmu_nlp_ssd/zhangbingyuan/benchmark/MultiPL-E/evaluation/src/main.py", line 44, in cached_eval_script result_dict = eval_string_script(problem["language"], program) File "/mmu_nlp_ssd/zhangbingyuan/benchmark/MultiPL-E/evaluation/src/containerized_eval.py", line 78, in eval_string_script result = eval_script(Path(f.name)) File "/mmu_nlp_ssd/zhangbingyuan/benchmark/MultiPL-E/evaluation/src/eval_rust.py", line 13, in eval_script build = subprocess.run(["rustc", path, "-o", basename], capture_output=True, timeout=15) File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/subprocess.py", line 503, in run with Popen(popenargs, **kwargs) as process: File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/subprocess.py", line 971, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/nlp_group/zhangbingyuan/anaconda/install_location/envs/autods/lib/python3.10/subprocess.py", line 1863, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'rustc'

May I ask what the problem is?

arjunguha commented 2 days ago

You're trying to run a Rust benchmark and Rust is not installed (rustc).