search

chnsh / DCRNN_PyTorch

Diffusion Convolutional Recurrent Neural Network Implementation in PyTorch
MIT License
454 stars 114 forks source link

Double sigmoid inside RNNCell #12

Closed razvanc92 closed 3 years ago

razvanc92 commented 4 years ago

I was looking at the implementation of the DCGRUCell, and I've spotted something out of order. If we are using 'fc' for U and R gates we are going to apply the sigmoid twice (line 121 and 96). Is this how it's intended to work, or is there a bug?

yuqirose commented 3 years ago

could you share a screenshot?

razvanc92 commented 3 years ago

@yuqirose Sure. If we look at the following image on line 96 the value is sigmoid(fn(...)).

ss1

Now if we look at the next picture (line 121) we are applying again sigmoid. ss2

Am I missing something?

yuqirose commented 3 years ago

@razvanc92 line 121 is applied to Wx, line 96 is applied to f(Wx) + b

razvanc92 commented 3 years ago

@yuqirose Thank you for your replay. Following the code we are doing sigmoid(sigmoid(Wx) + b) which is different from the original implementation of GRU which uses sigmoid(Wx +b).