How to improve performance?

[jamesqo]

I have a 33MB corpus and I ran this command:

python3 train.py --data_file=data/creepypasta.txt --hidden_size=512 --num_layers=2 --num_unrollings=64 --dropout=0.5 --verbose=1

It outputs an additional +0.4% progress every 110 seconds or so, which indicates that it'll take more than a week to complete 25 epochs. How can I speed this up?

[crazydonkey200]

Hi, there are several things you could do to accelerate training: reduce the hidden size; reduce the number of layers; reduce the num_unrollings; reduce the dropout.

In general, I would suggest trying something simple first. For example, maybe python3 train.py --data_file=data/creepypasta.txt --hidden_size=128 --num_layers=1 --num_unrollings=32 --dropout=0.0 --verbose=1 and then increase the complexity and regularization if the result is not good enough.