Request for Sequence Concatenation

[coopie]

What the title says - I would love to be able to pad a sequence of tensors with some zero tensors of equivalent shape. This would help with building 'look-behind' input for an RNN, where the first few inputs to the RNN need to have some input representing 'the sequence hasn't started yet'.

I would be happy to help in writing this, if someone could give me a few pointers.

Thanks

[Yorwba]

I had a need for sequence concatenation, too. Here is what I came up with after looking at the existing implementations in blocks.py:

import tensorflow_fold as td
from tensorflow_fold.blocks import result_types as tdt

class SequenceConcat(td.Block):
  """A block that concatenates its input sequences.

  The input type may be a Tuple or Sequence of Sequences
  """

  def __init__(self, name=None):
    super(SequenceConcat, self).__init__(name=name)

  _expected_input_types = (tdt.PyObjectType, tdt.TupleType, tdt.SequenceType)

  def _update_input_type(self):
    if isinstance(self.input_type, tdt.BroadcastSequenceType):
        raise TypeError('cannot concatenate elements of an infinite sequence: %s' %
                      self.input_type)
    self.set_output_type(td.blocks.blocks._infer_element_type(self.input_type))

  def _evaluate(self, _, x):
    return sum(x, [])

It could probably use some more safety checks, but maybe this works for you.