search

tensorflow / agents

TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.
Apache License 2.0
2.77k stars 714 forks source link

Network._build assumes input array spec is bounded #189

Open ericzhao28 opened 5 years ago

ericzhao28 commented 5 years ago

When defining a custom network, the Network class attempts to build by calling tensor_spec.sample_spec_nest to sample an input. Line 134 in tf_agents/network/network.py:

def _build(self):
    if not self.built and self.input_tensor_spec is not None:
      random_input = tensor_spec.sample_spec_nest(
          self.input_tensor_spec, outer_dims=(1,))
      step_type = tf.expand_dims(time_step.StepType.FIRST, 0)
      self.__call__(random_input, step_type, None)

However, sample_spec_nest attempts to convert each part of the input tensor spec to a BoundedTensorSpec. Line 331 in tf_agents/specs/tensor_spec.py:

    elif isinstance(spec, (TensorSpec, BoundedTensorSpec)):
      spec = BoundedTensorSpec.from_spec(spec)
      return sample_bounded_spec(
          spec, outer_dims=outer_dims, seed=seed_stream())

This results in a TypeError when a custom network consumes a (for example) Boolean array as part of its input. For instance:

  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tf_agents/networks/network.py", line 179, in trainable_variables
    self._build()
  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tf_agents/networks/network.py", line 135, in _build
    self.input_tensor_spec, outer_dims=(1,))
  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tf_agents/specs/tensor_spec.py", line 338, in sample_spec_nest
    return tf.nest.map_structure(sample_fn, structure)
  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tensorflow_core/python/util/nest.py", line 536, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tensorflow_core/python/util/nest.py", line 536, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tf_agents/specs/tensor_spec.py", line 332, in sample_fn
    spec = BoundedTensorSpec.from_spec(spec)
  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tensorflow_core/python/framework/tensor_spec.py", line 258, in from_spec
    minimum = getattr(spec, "minimum", dtype.min)
  File "/home/em/projects/ActiveDialogue/venv/lib/python3.6/site-packages/tensorflow_core/python/framework/dtypes.py", line 186, in min
    raise TypeError("Cannot find minimum value of %s." % self)
TypeError: Cannot find minimum value of <dtype: 'bool'>

Is there a recommended workaround for this issue?

RR-28023 commented 3 years ago

Hi @ericzhao28, I've run into the same issue. Did you find a workaround?