(This repository is no longer being maintained.) Code for Deep RL from Human Preferences [Christiano et al]. Plus a webapp for efficiently collecting human feedback.
Which tensorflow version should I need to use? I tested:
python rl_teacher/teach.py -e Breakout-v0 -n synth-test -p synth -l 300
and I get this error:
Using TensorFlow backend. Setting things up... Starting random rollouts to generate pretraining segments. No learning will take place... WARN: <class 'rl_teacher.envs.PixelEnvViewer'> doesn't implement 'reset' method, but it implements deprecated '_reset' method. Collected 40/76 segments Successfully collected 76 segments 2019-01-14 12:57:52.990717: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA Traceback (most recent call last): File "rl_teacher/teach.py", line 155, in <module> main() File "rl_teacher/teach.py", line 99, in main n_pretrain_labels, args.clip_length, args.stacked_frames, args.workers) File "/home/adminusl/rl-teacher-atari/rl_teacher/reward_models.py", line 112, in __init__ self.graph = self._build_model() File "/home/adminusl/rl-teacher-atari/rl_teacher/reward_models.py", line 135, in _build_model net = FullyConnectedMLP(self.obs_shape, self.act_shape) File "/home/adminusl/rl-teacher-atari/rl_teacher/nn.py", line 17, in __init__ self.model.add(Dense(h_size, input_dim=input_dim)) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/sequential.py", line 165, in add layer(x) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/base_layer.py", line 431, in __call__ self.build(unpack_singleton(input_shapes)) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/layers/core.py", line 866, in build constraint=self.kernel_constraint) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/base_layer.py", line 249, in add_weight weight = K.variable(initializer(shape), File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/initializers.py", line 218, in __call__ dtype=dtype, seed=self.seed) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 4139, in random_uniform dtype=dtype, seed=seed) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/ops/random_ops.py", line 240, in random_uniform shape, dtype, seed=seed1, seed2=seed2) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/ops/gen_random_ops.py", line 449, in _random_uniform name=name) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 609, in _apply_op_helper param_name=input_name) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 60, in _SatisfiesTypeConstraint ", ".join(dtypes.as_dtype(x).name for x in allowed_list))) TypeError: Value passed to parameter 'shape' has DataType float32 not in list of allowed values: int32, int64
How can I fix this issue?
malob
commented
5 years ago
Which tensorflow version should I need to use? I tested:
python rl_teacher/teach.py -e Breakout-v0 -n synth-test -p synth -l 300and I get this error:Using TensorFlow backend. Setting things up... Starting random rollouts to generate pretraining segments. No learning will take place... WARN: <class 'rl_teacher.envs.PixelEnvViewer'> doesn't implement 'reset' method, but it implements deprecated '_reset' method. Collected 40/76 segments Successfully collected 76 segments 2019-01-14 12:57:52.990717: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA Traceback (most recent call last): File "rl_teacher/teach.py", line 155, in <module> main() File "rl_teacher/teach.py", line 99, in main n_pretrain_labels, args.clip_length, args.stacked_frames, args.workers) File "/home/adminusl/rl-teacher-atari/rl_teacher/reward_models.py", line 112, in __init__ self.graph = self._build_model() File "/home/adminusl/rl-teacher-atari/rl_teacher/reward_models.py", line 135, in _build_model net = FullyConnectedMLP(self.obs_shape, self.act_shape) File "/home/adminusl/rl-teacher-atari/rl_teacher/nn.py", line 17, in __init__ self.model.add(Dense(h_size, input_dim=input_dim)) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/sequential.py", line 165, in add layer(x) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/base_layer.py", line 431, in __call__ self.build(unpack_singleton(input_shapes)) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/layers/core.py", line 866, in build constraint=self.kernel_constraint) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/engine/base_layer.py", line 249, in add_weight weight = K.variable(initializer(shape), File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/initializers.py", line 218, in __call__ dtype=dtype, seed=self.seed) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 4139, in random_uniform dtype=dtype, seed=seed) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/ops/random_ops.py", line 240, in random_uniform shape, dtype, seed=seed1, seed2=seed2) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/ops/gen_random_ops.py", line 449, in _random_uniform name=name) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 609, in _apply_op_helper param_name=input_name) File "/home/adminusl/anaconda3/envs/py35atari/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 60, in _SatisfiesTypeConstraint ", ".join(dtypes.as_dtype(x).name for x in allowed_list))) TypeError: Value passed to parameter 'shape' has DataType float32 not in list of allowed values: int32, int64How can I fix this issue?