search

eureka-research / Eureka

Official Repository for "Eureka: Human-Level Reward Design via Coding Large Language Models" (ICLR 2024)
https://eureka-research.github.io/
MIT License
2.73k stars 244 forks source link

subprocess.POPEN waits forever #13

Open SiqiLii opened 8 months ago

SiqiLii commented 8 months ago

I was running eureka.py and have output:

python eureka.py env=shadow_hand sample=4 iteration=2 model=gpt-4-0314 [2023-10-24 23:13:24,730][root][INFO] - Workspace: /home/123/isaacgym/python/Eureka/eureka/outputs/eureka/2023-10-24_23-13-24 [2023-10-24 23:13:24,730][root][INFO] - Project Root: /home/123/isaacgym/python/Eureka/eureka [2023-10-24 23:13:24,730][root][INFO] - Using LLM: gpt-4-0314 [2023-10-24 23:13:24,730][root][INFO] - Task: ShadowHand [2023-10-24 23:13:24,730][root][INFO] - Task description: to make the shadow hand spin the object to a target orientation [2023-10-24 23:13:24,758][root][INFO] - Iteration 0: Generating 4 samples with gpt-4-0314 [2023-10-24 23:14:01,699][root][INFO] - Iteration 0: Prompt Tokens: 1735, Completion Tokens: 1254, Total Tokens: 2989 [2023-10-24 23:14:01,699][root][INFO] - Iteration 0: Processing Code Run 0 [2023-10-24 23:14:13,181][root][INFO] - Iteration 0: Code Run 0 successfully training! [2023-10-24 23:14:13,181][root][INFO] - Iteration 0: Processing Code Run 1 [2023-10-24 23:14:26,009][root][INFO] - Iteration 0: Code Run 1 successfully training! [2023-10-24 23:14:26,009][root][INFO] - Iteration 0: Processing Code Run 2 [2023-10-24 23:14:26,009][root][INFO] - Iteration 0: Code Run 2 cannot parse function signature! [2023-10-24 23:14:26,009][root][INFO] - Iteration 0: Processing Code Run 3 [2023-10-24 23:14:40,169][root][INFO] - Iteration 0: Code Run 3 successfully training!

Then the process got stuck at rl.communication() and didn't move forward anymore

with Traceback: Traceback (most recent call last): File "eureka.py", line 397, in main() File "/home/.local/lib/python3.8/site-packages/hydra/main.py", line 94, in decorated_main _run_hydra( File "/home/.local/lib/python3.8/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra _run_app( File "/home/.local/lib/python3.8/site-packages/hydra/_internal/utils.py", line 457, in _run_app run_and_report( File "/home/.local/lib/python3.8/site-packages/hydra/_internal/utils.py", line 220, in run_and_report return func() File "/homer/.local/lib/python3.8/site-packages/hydra/_internal/utils.py", line 458, in lambda: hydra.run( File "/home/.local/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 119, in run ret = run_job( File "/home/.local/lib/python3.8/site-packages/hydra/core/utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) File "eureka.py", line 211, in main rl_run.communicate() File "/home/anaconda3/envs/HarryVae/lib/python3.8/subprocess.py", line 1020, in communicate self.wait() File "/home/anaconda3/envs/HarryVae/lib/python3.8/subprocess.py", line 1083, in wait return self._wait(timeout=timeout) File "/home/anaconda3/envs/HarryVae/lib/python3.8/subprocess.py", line 1822, in _wait (pid, sts) = self._try_wait(0) File "/home/anaconda3/envs/HarryVae/lib/python3.8/subprocess.py", line 1780, in _try_wait (pid, sts) = os.waitpid(self.pid, wait_flags)