On encoder-decoder implementation [Feature Request]

[ustctf-zz]

I'm wondering where there're any CNTK implementations for the encoder-decoder framework widely adopted in quite a lot of NLP sequence learning tasks, e.g., statistical machine translation in this paper (Neural Machine Translation by Jointly Learning to Align and Translate). If not, are you planning to implement this?

Thanks, Fei

[frankseide]

We are working on it. I will commit a few core building blocks hopefully tomorrow (currently in branch fseide/pack), then there will be three missing pieces:

• reading infrastructure that can deliver two (or more) inputs that consist of sequences with different length. This is being worked on (and will also, e.g., enable image captioning). There is also a workaround.
• attention models require a 2D storage. A fixed-length attention window can already be done.
• beam-search decoder. Greedy decoding can already be done.

I plan to set up a simple example for letter-to-sound mapping (simpler than full-scale translation). Stay tuned.

[ustctf-zz]

Thanks! Looking forward to the feature release!

[wolfma61]

assuming future push/commit will address this, for new issues please open a separat discussion