search

machine-intelligence / rl-teacher-atari

(This repository is no longer being maintained.) Code for Deep RL from Human Preferences [Christiano et al]. Plus a webapp for efficiently collecting human feedback.
MIT License
27 stars 6 forks source link

Which version of python/tensorflow should I use? #6

Open Axxeption opened 7 years ago

Axxeption commented 7 years ago

Hi,

I try to run you code, if I try with python 3.5.2 I got this error:

Using TensorFlow backend. Traceback (most recent call last): File "rl_teacher/teach.py", line 155, in <module> main() File "rl_teacher/teach.py", line 20, in main multiprocessing.set_start_method('spawn') File "/home/axel/miniconda3/envs/rlteacher/lib/python3.5/multiprocessing/context.py", line 231, in set_start_method raise RuntimeError('context has already been set') RuntimeError: context has already been set

If I try with python 3.6, I got this error: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/home/axel/rl-teacher-atari/agents/ga3c/ga3c/ThreadDynamicAdjustment.py", line 89, in run self.enable_disable_components() File "/home/axel/rl-teacher-atari/agents/ga3c/ga3c/ThreadDynamicAdjustment.py", line 70, in enable_disable_components self.server.add_agent() File "/home/axel/rl-teacher-atari/agents/ga3c/ga3c/Server.py", line 68, in add_agent self.agents[-1].start() File "/usr/lib/python3.5/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.5/multiprocessing/context.py", line 212, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.5/multiprocessing/context.py", line 274, in _Popen return Popen(process_obj) File "/usr/lib/python3.5/multiprocessing/popen_spawn_posix.py", line 33, in __init__ super().__init__(process_obj) File "/usr/lib/python3.5/multiprocessing/popen_fork.py", line 20, in __init__ self._launch(process_obj) File "/usr/lib/python3.5/multiprocessing/popen_spawn_posix.py", line 48, in _launch reduction.dump(process_obj, fp) File "/usr/lib/python3.5/multiprocessing/reduction.py", line 59, in dump ForkingPickler(file, protocol).dump(obj) TypeError: can't pickle _thread.lock objects So I don't know wich version to use. I tried installing it with pip, and this was succeed.

henrykmichalewski commented 6 years ago

With python 3.5 I am getting something similar 

Traceback (most recent call last):
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/home/henryk.michalewski/montezuma/rl-teacher-atari/agents/ga3c/ga3c/ThreadDynamicAdjustment.py", line 89, in run
    self.enable_disable_components()
  File "/home/henryk.michalewski/montezuma/rl-teacher-atari/agents/ga3c/ga3c/ThreadDynamicAdjustment.py", line 70, in enable_disable_components
    self.server.add_agent()
  File "/home/henryk.michalewski/montezuma/rl-teacher-atari/agents/ga3c/ga3c/Server.py", line 68, in add_agent
    self.agents[-1].start()
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/multiprocessing/context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/multiprocessing/context.py", line 274, in _Popen
    return Popen(process_obj)
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/multiprocessing/popen_spawn_posix.py", line 33, in __init__
    super().__init__(process_obj)
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/multiprocessing/popen_spawn_posix.py", line 48, in _launch
    reduction.dump(process_obj, fp)
  File "/home/henryk.michalewski/anaconda2/envs/monteconda/lib/python3.5/multiprocessing/reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects
henrykmichalewski commented 6 years ago

A partial resolution to the issue

    # Tensorflow is not fork-safe, so we must use spawn instead
    # https://github.com/tensorflow/tensorflow/issues/5448#issuecomment-258934405
    multiprocessing.set_start_method('spawn')

This is a part of teach.py. I commented the last line and the above error disappears. Apparently the official implementation of GA3C from Nvidia does not have a call multiprocessing.set_start_method('spawn') and still works. I am assuming that authors of rl-teacherconsciously added this line to avoid some other troubles, hence this fix may appear not satisfactory. Hopefully we hear something from the authors soon :)

amanjhunjhunwala commented 6 years ago

Had same problem, Complete resolution : Use gym version 0.9 Tested with Tensorflow==1.5

Anmol6 commented 6 years ago

@amanjhunjhunwala I tried those versions but still can't get it to work. Could you provide .yml file of your conda environment (or Dockerfile if you're not using conda)?

ZhanPython commented 5 years ago

Which tensorflow version should I need to use? I tested: python rl_teacher/teach.py -e Breakout-v0 -n synth-test -p synth -l 300 and I get this error: Using TensorFlow backend. Setting things up... Starting random rollouts to generate pretraining segments. No learning will take place... WARN: <class 'rl_teacher.envs.PixelEnvViewer'> doesn't implement 'reset' method, but it implements deprecated '_reset' method. Collected 40/76 segments Successfully collected 76 segments 2019-01-14 12:57:52.990717: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA Traceback (most recent call last): File "rl_teacher/teach.py", line 155, in <module> main() File "rl_teacher/teach.py", line 99, in main n_pretrain_labels, args.clip_length, args.stacked_frames, args.workers) File "/home/adminusl/rl-teacher-atari/rl_teacher/reward_models.py", line 112, in __init__ self.graph = self._build_model() File "/home/adminusl/rl-teacher-atari/rl_teacher/reward_models.py", line 135, in _build_model net = FullyConnectedMLP(self.obs_shape, self.act_shape) File "/home/adminusl/rl-teacher-atari/rl_teacher/nn.py", line 17, in __init__ self.model.add(Dense(h_size, input_dim=input_dim)) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/sequential.py", line 165, in add layer(x) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/base_layer.py", line 431, in __call__ self.build(unpack_singleton(input_shapes)) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/layers/core.py", line 866, in build constraint=self.kernel_constraint) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/base_layer.py", line 249, in add_weight weight = K.variable(initializer(shape), File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/initializers.py", line 218, in __call__ dtype=dtype, seed=self.seed) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 4139, in random_uniform dtype=dtype, seed=seed) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/ops/random_ops.py", line 240, in random_uniform shape, dtype, seed=seed1, seed2=seed2) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/ops/gen_random_ops.py", line 449, in _random_uniform name=name) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 609, in _apply_op_helper param_name=input_name) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 60, in _SatisfiesTypeConstraint ", ".join(dtypes.as_dtype(x).name for x in allowed_list))) TypeError: Value passed to parameter 'shape' has DataType float32 not in list of allowed values: int32, int64 How can I fix this issue?

malob commented 5 years ago

Just wanted to leave a quick note to say that we are no longer maintaining this project, and so won't be responding to issues etc. I'll leave this issue open however, so others who want to use the project can help each other solve issues etc.