search

vlfeat / matconvnet

MatConvNet: CNNs for MATLAB
Other
1.4k stars 753 forks source link

can i use batch normalization in RNN in matconvnet? #983

Closed leoyu800 closed 5 years ago

leoyu800 commented 7 years ago

hi, every one. I'm new in matconvnet. I have some problems using batch normalization when I share weights in different conv layers. It sounds like a RNN using DAG. I noted that bn seem doesn't work in RNN, but these days I found there is a new approach using batch normalization in RNN. Does matconvnet include this approach or it would be a future feature. Sorry for my pool english, any advise would be great!

bazilas commented 7 years ago

The classic BN works fine on vanilla RNNs. check out the following project to see the configuration:

https://github.com/ox-vgg/keypoint_detection

leoyu800 commented 7 years ago

Hi, bazilas. Thanks for you response. I see the project about keypoint_detection. His network is divided into two parts: sequential part and recurrent part. He uses BN in sequential part but not recurrent part. I can use BN successfully in simple sequential net but have problem in recurrent part, any idea?

bazilas commented 7 years ago

you could use BN on the recurrent part, as in:

https://github.com/ox-vgg/keypoint_detection/blob/master/dagnetworks/initialize3ObjeRecFusion.m#L445-L462