Error when trying to train num_layer > 1

[dimeldo]

Hi,

When I set:

cxt_cell_size = 150  # context encoder hidden size
sent_cell_size = 75  # utterance encoder hidden size
dec_cell_size = 100  # response decoder hidden size
num_layer = 4  # number of context RNN layers

I get the following error:

ValueError: Trying to share variable model/contextRNN/rnn/multi_rnn_cell/cell_0/gru_cell/gates/kernel, but specified shape (300, 300) and found shape (302, 300).

[dimeldo]

Okay, I found the solution for this. The problem with the syntax you currently have is that it tries to reuse the same cell object num_layer times, so we run into issues with sharing variables. So I defined each cell uniquely and avoid accidentally sharing variables. This is the code I've used that works:

    @staticmethod
    def get_rnncell(cell_type, cell_size, keep_prob, num_layer):
        cells = []
        for _ in range(num_layer):
            if cell_type == "gru":
                cell = rnn_cell.GRUCell(cell_size)
            else:
                cell = rnn_cell.LSTMCell(cell_size, use_peepholes=False, forget_bias=1.0)

            if keep_prob < 1.0:
                cell = rnn_cell.DropoutWrapper(cell, output_keep_prob=keep_prob)

            cells.append(cell)

        if num_layer > 1:
            cell = rnn_cell.MultiRNNCell(cells, state_is_tuple=True)
        else:
            cell = cells[0]

        return cell

I would like your verification that this is the right solution. And then you can close this issue.

(B.T.W. these parameters seems to work badly, don't use them)

[snakeztc]

@dimeldo Thanks! you solution is correct. I push the changes with your suggested solution..