search

bquast / rnn

Recurrent Neural Networks in R
https://qua.st/rnn
73 stars 28 forks source link

equation bugs #31

Open alongwithyou opened 6 years ago

alongwithyou commented 6 years ago

I found delta6=1-delta5,but there are equation: f(zt)=1-zt, delta6 to delata5 will be delta5*(1-zt)' and will be equal to -zt, that means : delta6=-delta5, not 1-delta5, is it right? thanks.

mratsim commented 6 years ago

The equation for delta 6 is indeed wrong.

For manually implemented backprop, tests should include numerical gradient check to catch those issues.