seungjaeryanlee / agents

TF-Agents is a library for Reinforcement Learning in TensorFlow
Apache License 2.0
1 stars 0 forks source link

tf.clip_by_value results in an error in _init_rnd_normalizer #12

Closed seungjaeryanlee closed 5 years ago

seungjaeryanlee commented 5 years ago

Upon searching online, it seems like this happens when the memory is too small?

2019-07-25 04:09:29.063615: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-07-25 04:09:29.088399: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2712000000 Hz
2019-07-25 04:09:29.088872: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ec3676cd00 executing computations on platform Host. Devices:
2019-07-25 04:09:29.088950: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
I0725 04:09:29.253317 139884285708032 parallel_py_environment.py:81] Spawning all processes.
I0725 04:09:31.808124 139884285708032 parallel_py_environment.py:88] All processes started.
W0725 04:09:40.928066 139884285708032 deprecation.py:323] From /home/rlee/git/agents/tf_agents/replay_buffers/tf_uniform_replay_buffer.py:540: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
Traceback (most recent call last):
  File "tf_agents/agents/ppo/examples/v2/train_eval_atari.py", line 335, in <module>
    app.run(main)
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "tf_agents/agents/ppo/examples/v2/train_eval_atari.py", line 330, in main
    num_eval_episodes=FLAGS.num_eval_episodes)
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/gin/config.py", line 1032, in wrapper
    utils.augment_exception_message_and_reraise(e, err_str)
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/gin/utils.py", line 49, in augment_exception_message_and_reraise
    six.raise_from(proxy.with_traceback(exception.__traceback__), None)
  File "<string>", line 3, in raise_from
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/gin/config.py", line 1009, in wrapper
    return fn(*new_args, **new_kwargs)
  File "tf_agents/agents/ppo/examples/v2/train_eval_atari.py", line 247, in train_eval
    tf_agent._init_rnd_normalizer(experience=trajectories)
  File "/home/rlee/git/agents/tf_agents/agents/ppo/rndppo_agent.py", line 330, in _init_rnd_normalizer
    intrinsic_rewards, _ = self.rnd_loss(time_steps, debug_summaries=self._debug_summaries)
  File "/home/rlee/git/agents/tf_agents/agents/ppo/rndppo_agent.py", line 777, in rnd_loss
    self._observation_clip_value)
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
    return target(*args, **kwargs)
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/tensorflow_core/python/ops/clip_ops.py", line 83, in clip_by_value
    t_min = math_ops.minimum(values, clip_value_max)
  File "/home/rlee/anaconda3/envs/gsoc/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py", line 6443, in minimum
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: Could not find valid device for node.
Node:{{node Minimum}}
All kernels registered for op Minimum :
  device='CPU'; T in [DT_INT64]
  device='CPU'; T in [DT_INT32]
  device='CPU'; T in [DT_DOUBLE]
  device='CPU'; T in [DT_BFLOAT16]
  device='CPU'; T in [DT_HALF]
  device='CPU'; T in [DT_FLOAT]
  device='XLA_CPU'; T in [DT_FLOAT, DT_DOUBLE, DT_INT32, DT_INT64, DT_BFLOAT16, DT_HALF]
  device='XLA_CPU_JIT'; T in [DT_FLOAT, DT_DOUBLE, DT_INT32, DT_INT64, DT_BFLOAT16, DT_HALF]
 [Op:Minimum]
  In call to configurable 'train_eval' (<function train_eval at 0x7f39591f71e0>)
seungjaeryanlee commented 5 years ago

Solved by casting uint8 to float32

tf.cast(time_steps.observation, tf.float32)

However, this should have been normalized?