Adding Dense Layers of Different Sizes

[collinskatie]

Hi,

I'm having trouble adding dense layers of with different neuron numbers. For instance, if I would like 3 hidden layers of widths 300, 100, 30 - is there a way to initialize a pysster model to handle this?

Thank you for your help!

[budach]

Hi, this is currently not possible, all dense layers always have the same number of neurons. I found that for sequence-related tasks, the number (and size) of dense layers rarely makes a difference (it also keeps the params configuration simple). This always depends on the input data set of course.

I can change the params configuration a little bit (just for testing purposes for now), so that you are able to do this. That shouldn't take too long, I will try to do it tomorrow.

[collinskatie]

Thank you so much!

[budach]

With commit https://github.com/budach/pysster/commit/075e34b3a7b1860917b26a6db1e0b6411df2c86a the params configuration should now be more flexible. Right now the change is not yet on PyPI, so please install the Github version. The change is also not yet documented anywhere except here.

Previously you would build a model like this (it still works) to create a model with 3 three dense layers which all have 100 neurons:

params = {"dense_num": 3, "neuron_num": 100}
model = Model(params, data)

Now you can also specify params of all individual dense layers by providing a tuple. The following will create a model in which the first dense layer has 300 neurons, the second 100 and the third 30:

params = {"dense_num": 3, "neuron_num": (300, 100, 30)}
model = Model(params, data)

This works for all parameters that a layer depends on. For dense layers those are: "neuron_num", "dropout_dense". For convolutional layers those are: "kernel_num", "kernel_len", "pool_size", "pool_stride", "dropout_conv". For RNN layers those are: "rnn_units", "rnn_bidirectional", "rnn_dropout_recurrent", "rnn_dropout_input".

Another example: the following model has two convolutional layers (the first layer has 10 kernels of length 30, the second layer 20 kernels of length 3) and two dense layers (first dense layer has 100 neurons, the second 10).

params = {"conv_num": 2, "kernel_num": (10, 20), "kernel_len": (30, 3),
          "dense_num": 2, "neuron_num": (100, 10)}
model = Model(params, data)

I hope this helps!

[collinskatie]

Perfect! That helps tremendously. Thank you!

[budach]

This is already available since version 1.2.1 and also part of the docs know. Closing.