uyaseen
commented
8 years ago Hi Yang,
Glad to know that you found the code helpful.
The distinction between cell and blocks eroded over time, most of modern LSTM architecuters have one cell per block, (which in my opinion is simple), regarding your question, "why not much attention was paid to this difference", I am afraid I might not have a very clear answer, I would say:
-> Do we have any empirical evidence which suggests that LSTMs with multiple cells architecture works better than "one cell per block" architecture ? [I am not aware of any such evidence, If we don't have any such evidence then people will prefer less cumbersome model]; Also same applies for peep-hole connections, some people don't use them as they don't find them very helpful -> Why not increase the size of the memory (capacity) of a cell instead of adding more cells in a block ? (I would prefer increasing the memory, it's simple & more interpretable ; and to me it looks like they are equivalent as well, since both architectures are using the same gates ["cells within a block use the same gates"]) -> Simple is always better (GRUs are simple version of LSTMs and have almost equivalent performance in many task, therefore, are very popular these days)
I hope above explanation helps a bit, [1] explains the difference between various LSTM architectures.
xy0806
son20112074
DongGuangchang
Dear Yaseen, thanks for your clean code. As you know, there have the conceptions 'LSTM block' and 'LSTM cell'. But in a lot of LSTM example codes, including yours, there seems to be no attention was paid to this difference. In the codes, only cells are created, while no blocks. After reading and thinking about this problem, I got the conclusion that: the LSTM with m blocks with n cells and the LSTM with one block with m*n cells are actually the same. Then, how do you think about this problem and could you give me any hints about this issue?
Thanks, Xin Yang