AttributeError: Can't pickle local object 'check_correctness.<locals>.unsafe_execute'

[tianzhaotju]

When I run "evaluate_functional_correctness sample.jsonl --problem_file=problem.jsonl", it has the following problem.

Can u help me? thx Detail log.

---

Reading samples... 1it [00:00, 2118.34it/s] Running test suites... 0%| | 0/1 [00:00<?, ?it/s] Traceback (most recent call last): File "/Users/user/opt/anaconda3/envs/py38/bin/evaluate_functional_correctness", line 33, in sys.exit(load_entry_point('human-eval', 'console_scripts', 'evaluate_functional_correctness')()) File "/Users/user/opt/anaconda3/envs/py38/bin/evaluate_functional_correctness", line 25, in importlib_load_entry_point return next(matches).load() File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/importlib/metadata.py", line 77, in load module = import_module(match.group('module')) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 843, in exec_module File "", line 219, in _call_with_frames_removed File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 28, in sys.exit(main()) File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 25, in main fire.Fire(entry_point) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, *kwargs) File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 20, in entry_point results = evaluate_functional_correctness(sample_file, k, n_workers, timeout, problem_file) File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluation.py", line 75, in evaluate_functional_correctness result = future.result() File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.get_result() File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/_base.py", line 389, in get_result raise self._exception File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(self.args, **self.kwargs) File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/execution.py", line 73, in check_correctness p.start() File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'check_correctness..unsafe_execute'

[zhouyangen]

你可以参考18楼的回复

[emrgnt-cmplxty]

Seeing the same error, what is the fix?

[Fazziekey]

Same problem

[Fazziekey]

@tianzhaotju @emrgnt-cmplxty I have fixed this problem in my repo with pull request https://github.com/openai/human-eval/pull/30

[maximw99]

Hello Devs, i have encountered this issue too. Well for me the suggested fix does not work at all. My workaround is to use a Linux Distro under WSL-2 or just by itself. For me it straight clicks, and everything works fine. I am not sure why the problem is present only on my windows system. Hope I could help.

[jckpn]

Also getting this error on macOS Sonoma, but it runs fine in a Windows VM.