Formula to calculate the number of parameters

karpathy / char-rnn

Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch

11.52k stars 2.58k forks source link

Formula to calculate the number of parameters #187

Open zhenyulin opened 7 years ago

zhenyulin commented 7 years ago

I'm relatively new to Deep Learning, but found this repo extremely interesting and a really good way to learn the topic.

I was wondering if there's a formula to calculate the number of parameters generated by the settings of rnn, so that to match the scale of the input data size?

A quick search has directed me here, but apparently it doesn't match the result provided in the example.

Would be great if someone can provide some further guidance here. Thanks!

chestnuut commented 7 years ago

one useful reference is here Suppose input size is m, hidden size is n, output size is k. Then the parameters you need to train is: n^2 + mn + kn

Note this is only for a basic RNN with one hidden layer.