hunse
commented
2 years ago I've made a few modifications. The main one is that the bias is no longer a subsidiary of the kernel, because a user may want to have a bias for the recurrent kernel when they have the encoding kernel turned off.
I also added a separate initialization and regularization term for the bias, since these are typically treated separately (e.g. in Dense layers), and one usually wants a different initializer for the biases (i.e. zeros) than for the weights. The use_bias argument is now a little redundant, in that we could accomplish this by just having the user set bias_initializer to None if they don't want a bias (this is what we do with the kernel). However, I kept it because we want the bias to be off by default, but if the user wants to turn the bias on, it's nice to have a sensible default for the initializer (i.e. zeros). Otherwise, any user that wants to turn the bias on would need to know what initializer they want to use (though we could help them out in the docstring by suggesting "zeros").
vsheska
Added options to include a bias vector, and regularizers to the kernels