How do I implement a many-to-many RNN for language modeling?

[camcam3]

Most of the examples I read in Keras such as the char_rnn implement RNNs where the prediction is made at the end only. For language modeling, I would like to predict y after each step, such as the one described in this paper http://arxiv.org/pdf/1409.2329v5.pdf as implemented by TensorFlow https://www.tensorflow.org/versions/master/tutorials/recurrent/index.html.

[fchollet]

You use return_sequences=True in your RNN layer(s). It's in the docs.