Performance seems to be better with LSTMs rather than RNNs, however this strays away from the DS2 structure. GRUs are very finnicky to get to converge whereas LSTMs are much more robust when it comes to convergence.
If the validation scores increase using LSTMs I'll add an option to switch the RNNs to BLSTMs from the cuDNN package.
SeanNaren
commented
8 years ago
Performance seems to be better with LSTMs rather than RNNs, however this strays away from the DS2 structure. GRUs are very finnicky to get to converge whereas LSTMs are much more robust when it comes to convergence.
If the validation scores increase using LSTMs I'll add an option to switch the RNNs to BLSTMs from the cuDNN package.