search

jfrancis71 / CognitoNet

Retired: Feedforward Convolutional Neural Network for Mathematica
MIT License
13 stars 2 forks source link

Is Dropout implemented correctly? #2

Closed jfrancis71 closed 8 years ago

jfrancis71 commented 8 years ago

CIFAR-10 results showed only marginal improvement with dropout. Results on other datasets do not seem consistent with reported results in the academic community.

Is there a bug in the code? Slightly suspicious that the dropout mask is being passed correctly in the forward propogation phase.

Am tomorrow out of UK, but will look at this on my return (after 1 June).

Will only affect models being trained with dropout.

jfrancis71 commented 8 years ago

OK, have double checked. It hadn't quite migrated into CognitoNet from the previous repository. So the previous comment refers to that repository. However it does look as if it wasn't implemented quite correctly. Having said that implementing the bugfix doesn't seem to make much difference. My best guess is that although the forward propogated values were incorrect (i.e. we were forward propogating on the non dropped out version of the net) they were on average correct. Hence the fix doesn't seem to have made much difference.