redpanda-ai / Meerkat

Used for the Meerkat project
Other
1 stars 1 forks source link

Research QRNNs #989

Open speakerjohnash opened 7 years ago

speakerjohnash commented 7 years ago

"Despite lacking trainable recurrent layers, stacked QRNNs have better predictive accuracy than stacked LSTMs of the same hidden size. Due to their increased parallelism, they are up to 16 times faster at train and test time"

https://arxiv.org/abs/1611.01576

RNN is currently a 30th of the speed of the CNN per transaction. We have two updates on the horizon, one to double performance and then another to quadruple it. This may make those updates unnecessary if easy to implement.