wonder why not use the default LSTM

[allanj]

It seems you build an LSTM from the scratch

[yzhangcs]

As described in the paper:

we drop nodes in the LSTM layers (input and recurrent connections), applying the same dropout mask at every recurrent timestep (cf. the Bayesian dropout of Gal & Ghahramani (2015));