Sequence classification: GPU runs out of memory

[DavidNemeskey]

Maybe the behaviour of dynamic_rnn changed, or the author used a graphic card with more memory than me, but on a GTX 980 with 4GB, the sequence classification code doesn't run.

I think the reason is that the length of the longest review is around 2700, and 2700x300xbatch_size is too much for the card. It would be nice if the book addressed this issue.

I guess a solution would be to unroll the network to e.g. 50 steps, but still feed the batches... is it possible?

[danijar]

Hi David, I'm sorry I didn't get a notification about your issue. It's probably too late but in case someone finds this and is looking for a solution, there are several options:

• Swap memory. Set swap_memory=True in tf.nn.dynamic_rnn(). This moves activations computed during the forward pass from GPU to CPU, and move them back for the backward pass.
• Run on CPU. GPUs speed up deep learning because of their massive parallelism. However, (standard) RNNs operate on the input frame by frame, so that the benefit of using a GPU is actually not that big.
• Truncated backpropagation. As you suggested in the question. Split the sequences into chunks of equal length, for example 200 frames. To have an error signal, we let the network classify the sequence class from just the current chunk. It improves performance to store the last RNN state in a variable and pass it into the next chunk, so that it is preserved along a full sequence.
• More smaller layers. The number of weights of an RNN layer is quadratic in the layer size. Therefore, using 3 smaller layers often works similarly well as 1 big layer but has a better chance to fit into memory.

I hope this helps and please let me know if there are any other questions.