from example in doc (my brain bug or tff bug ?)

[benjaminderei]

Hello all,

I try to do this:

import tensorflow as tf
import tensorflow_fold as td

# Create RNN cells using the TensorFlow RNN library
char_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=16), 'char_cell')
word_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=32), 'word_cell')

# character LSTM converts a string to a word vector
char_lstm = (td.InputTransform(lambda s: [ord(c) for c in s]) >>
             td.Map(td.Scalar('int32') >>
                    td.Function(td.Embedding(128, 8))) >>
             td.RNN(char_cell))
# word LSTM converts a sequence of word vectors to a sentence vector.
# word_lstm = td.Map(char_lstm >> td.GetItem(1)) >> td.RNN(word_cell)

# ---------------------------------------------------------------------

sess = tf.InteractiveSession()

(
    td.Map(char_lstm >> td.GetItem(1)) >>
    td.RNN(word_cell)
).eval(["bon","ben"])

and i get:

(...)
/opt/conda/envs/python2/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.pyc in _linear(args, output_size, bias, bias_start, scope)
    729   # Calculate the total size of arguments on dimension 1.
    730   total_arg_size = 0
--> 731   shapes = [a.get_shape() for a in args]
    732   for shape in shapes:
    733     if shape.ndims != 2:

AttributeError: 'tuple' object has no attribute 'get_shape'

[coopie]

you need to set state_is_tuple to be false, so the cell and hidden state are concatenated together.

i.e

# Create RNN cells using the TensorFlow RNN library
char_cell = td.ScopedLayer(
    tf.contrib.rnn.BasicLSTMCell(num_units=16, state_is_tuple=False),
    'char_cell'
)
word_cell = td.ScopedLayer(
    tf.contrib.rnn.BasicLSTMCell(num_units=32, state_is_tuple=False),
    'word_cell'
)

The example should work. I'll leave it up to you to figure out what the output of the eval actually represents :)

[benjaminderei]

Thank you very much !!! no i can continue my « brain burning » :)

I think i will figure a lot of things before figuring the right one: Learning game !

[pklfz]

According to section 3.3 of hierarchical LSTMs, the word_lstm gets the hidden outputs of char_lstm as inputs. So, I think the correct way to construct a word_lstm cell with char_cell's state_is_tuple=True is here below:

import tensorflow as tf
import tensorflow_fold as td

# Create RNN cells using the TensorFlow RNN library
char_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=16), 'char_cell')
word_cell = td.ScopedLayer(tf.contrib.rnn.BasicLSTMCell(num_units=32), 'word_cell')

# character LSTM converts a string to a word vector
char_lstm = (td.InputTransform(lambda s: [ord(c) for c in s]) >>
             td.Map(td.Scalar('int32') >>
                    td.Function(td.Embedding(128, 8))) >>
             td.RNN(char_cell))
# word LSTM converts a sequence of word vectors to a sentence vector.
word_lstm = td.Map(char_lstm >> td.GetItem(1) >> td.GetItem(1)) >> td.RNN(word_cell)

sess = tf.InteractiveSession()

word_lstm.eval(["bon","ben"])

I add one more >> td.GetItem(1) behind >> td.GetItem(1), so that word_lstm can get the hidden_output of one char_lstm instead of (cell_status, hidden_output) of one.

Additionally, the content of test_hierarchical_rnn in blocks_test.py also need to be revised.

Wish this is helpful!

[benjaminderei]

Will Try :)

~~but are you sure ? if i remember the char_lstm_output[1] is just 32 float long... anyway i will check !~~

Yeah you right but with state_is_tuple=True ! I'm dumb 8~| :)

[pklfz]

@benjaminderei No. char_lstm_output[1] is an instance of LSTMStateTuple if you set state_is_tuple=True; otherwise, it is 32 float numpy array.

[benjaminderei]

I have edited my previous post :)

[pklfz]

@benjaminderei glad to help you.