Should tanh_out and sigm_out in residual_block share the same convolutional output

basveeling / wavenet

Keras WaveNet implementation

https://soundcloud.com/basveeling/wavenet-sample

1.06k stars 219 forks source link

Should tanh_out and sigm_out in residual_block share the same convolutional output #37

Closed lidalei closed 5 years ago

lidalei commented 5 years ago

Shouldn't tanh_out in https://github.com/basveeling/wavenet/blob/master/wavenet.py#L226 and sigm_out in https://github.com/basveeling/wavenet/blob/master/wavenet.py#L230 share the same convolutional output instead of two independent convolutional outputs?

lidalei commented 5 years ago

Also, this line https://github.com/basveeling/wavenet/blob/master/wavenet.py#L236 looks useless.

zhuyiche commented 5 years ago

read paper more carefully and compare to the other implementations before your open an issue

lidalei commented 5 years ago

Which message are you replying to? The paper says it uses a new kind of activation function. Using two convolutional layers and combining them afterwards is not equivalent to that. I will read the paper again and come back later (on a holiday).

basveeling commented 5 years ago

Hi Dalei,

If you believe that the gated activation is implemented incorrectly, please feel free to submit a pull request with some evidence that your proposed implementation is more faithful to the paper.

Thanks,

Bas