After about 3000s of runtime in teach.py, I get a GLFW exception. It's happened the last 3 times. I've noticed that after the crash, some video segments are still recorded and uploaded to Google Cloud, but not others. In once case, only the -right.mp4 vids were being recorded.
[2017-12-05 09:39:25,421] GLFW error: 65537, desc: b'The GLFW library is not initialized'
[2017-12-05 09:39:25,424] GLFW error: 65537, desc: b'The GLFW library is not initialized'
Copying media to gs://rl-teacher-walker-splits-1/8cc4f334-b515-44fd-8299-67619e280dae-right.mp4 @2906s
Copying media to gs://rl-teacher-walker-splits-1/ef8d7d30-8e34-46ef-b141-bc63c9380693-right.mp4 @2906s
Copying file:///tmp/rl_teacher_media/ef8d7d30-8e34-46ef-b141-bc63c9380693-left.mp4 [Content-Type=video/mp4]...
-------- Iteration 168 ----------
Average sum of true rewards per episode: -879.5909
Entropy: 8.3204
KL(old|new): 0.0005
Surrogate loss: -0.0022
Frames gathered: 8000
Frames gathered/second: 723
Time spent gathering rollouts: 11.06
Time spent updating weights: 4.40
Total time: 22:08.5
Copying file:///tmp/rl_teacher_media/8cc4f334-b515-44fd-8299-67619e280dae-right.mp4 [Content-Type=video/mp4]...
Copying file:///tmp/rl_teacher_media/ef8d7d30-8e34-46ef-b141-bc63c9380693-right.mp4 [Content-Type=video/mp4]...
\ [1 files][106.6 KiB/106.6 KiB]
Operation completed over 1 objects/106.6 KiB.
GCS OK, path= gs://rl-teacher-walker-splits-1/ef8d7d30-8e34-46ef-b141-bc63c9380693-left.mp4 >>> /ef8d7d30- @2912s
\ [1 files][ 3.2 KiB/ 3.2 KiB]
Operation completed over 1 objects/3.2 KiB.
\ [1 files][ 3.2 KiB/ 3.2 KiB]
Operation completed over 1 objects/3.2 KiB.
GCS OK, path= gs://rl-teacher-walker-splits-1/8cc4f334-b515-44fd-8299-67619e280dae-right.mp4 >>> /8cc4f334- @2913s
[2017-12-05 09:39:32,677] GLFW error: 65537, desc: b'The GLFW library is not initialized'
[2017-12-05 09:39:32,677] GLFW error: 65537, desc: b'The GLFW library is not initialized'
Exception ignored in: <bound method Env.__del__ of <TimeLimitTransparent<NeverDone<MjViewer<Walker2dEnv instance>>>>>
Traceback (most recent call last):
File "~/openai2/gym/gym/core.py", line 203, in __del__
self.close()
File "~/openai2/gym/gym/core.py", line 166, in close
self._close()
File "~/openai2/gym/gym/core.py", line 290, in _close
return self.env.close()
File "~/openai2/gym/gym/core.py", line 166, in close
self._close()
File "~/openai2/gym/gym/core.py", line 290, in _close
return self.env.close()
File "~/openai2/gym/gym/core.py", line 166, in close
self._close()
File "~/openai2/gym/gym/core.py", line 290, in _close
return self.env.close()
File "~/openai2/gym/gym/core.py", line 164, in close
self.render(close=True)
File "~/openai2/gym/gym/core.py", line 150, in render
return self._render(mode=mode, close=close)
File "~/openai2/gym/gym/envs/mujoco/mujoco_env.py", line 105, in _render
self._get_viewer().finish()
File "~/anaconda/envs/gym/lib/python3.5/site-packages/mujoco_py/mjviewer.py", line 325, in finish
glfw.destroy_window(self.window)
File "~/anaconda/envs/gym/lib/python3.5/site-packages/mujoco_py/glfw.py", line 809, in destroy_window
window_addr = ctypes.cast(ctypes.pointer(window),
TypeError: _type_ must have storage info
[2017-12-05 09:39:32,698] GLFW error: 65537, desc: b'The GLFW library is not initialized'
also sometime after the GLFW exception, I get these error messages, but they might be related
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
[...]
After about 3000s of runtime in
teach.py, I get aGLFW exception. It's happened the last 3 times. I've noticed that after the crash, some video segments are still recorded and uploaded to Google Cloud, but not others. In once case, only the-right.mp4vids were being recorded.Any ideas?
python rl_teacher/teach.py -p human --pretrain_labels 100 -e walker-n walker-splits-1 -t 16000stdout
also sometime after the
GLFWexception, I get these error messages, but they might be related