search

Neuroglycerin / neukrill-net-work

NDSB competition repository for scripting, note taking and writing submissions.
MIT License
0 stars 0 forks source link

Weight explosion in fully connected layer #67

Closed gngdb closed 9 years ago

gngdb commented 9 years ago

Looking at the results from the Analysing Network notebook we're getting massive numbers for the parameters in the weight matrix connected the convolutional to fully connected layers. Seems like this is probably something we should avoid. Will look into this using max pooling. Notes will be in the iterative design notebook.

gngdb commented 9 years ago

OK, this issue is badly titled. It's not an explosion in parameter values, just in the number of parameters due to max pooling problems.

gngdb commented 9 years ago

This can be solved by changing the max pooling options carefully. Recommend writing model in notebook and printing the model while changing pool shape and stride.

scottclowe commented 9 years ago

If you mess this up too badly then the model won't initialise due memory errors... there are more weights the the GPU memory has space for!