Standardize parameter structure

IDSIA / brainstorm

Fast, flexible and fun neural networks.

Other

1.3k stars 152 forks source link

Standardize parameter structure #22

Closed flukeskywalker closed 9 years ago

flukeskywalker commented 9 years ago

I think we should (or need to) rearrange the axes of the weight matrices. The convention should be that the first axis always corresponds to the number of units/neurons in the layer. This will:

Be compatible with parameter structures in Caffe etc.
Simplify code for writing weight modifiers etc.

This does not require too much work yet. If it sounds good I can do it.

untom commented 9 years ago

Is the 'first' the outer-most or the inner-most axis? (i.e., will the fully-connected weight matrix be #units * #inputs or vice-versa?)

flukeskywalker commented 9 years ago

Fully connected matrix will be #units * #inputs. Conv layer weights will be #filters * #num_input_filters * kernel_size.

untom commented 9 years ago

In that case yes, that sounds good!

Qwlouse commented 9 years ago

:+1:

flukeskywalker commented 9 years ago

Great. I've already changed FC, and will move RNN/LSTM too.