Initial values for weights and Biases in MLP Classifier

[jimmyroyer]

Hi, is it possible to change the initial values for the weights and biases? I think they are set in Lasagne but I don't know which initial values are used and if I can change them using "Classifier".

Thank you very much

[alexjc]

You can access parameters at low-level using get_parameters() and set_parameters(). It's Lasagne default, but it doesn't matter too much if you use batch normalization. There's no way to customize them except get/set.

[jimmyroyer]

Am I right that I can access those functions through the _backend of the classifier? Thank you very much

[jimmyroyer]

Also, when I try to use the option normalize="batch" I get the following error message. Any help will be very useful. Thanks 1

Traceback (most recent call last): File "", line 1, in File "C:\Anaconda\lib\site-packages\sknn\mlp.py", line 397, in fit return super(Classifier, self)._fit(X, yp, w) File "C:\Anaconda\lib\site-packages\sknn\mlp.py", line 213, in _fit X, y = self._initialize(X, y, w) File "C:\Anaconda\lib\site-packages\sknn\mlp.py", line 42, in _initialize return self._backend._initialize_impl(X, y, w) File "C:\Anaconda\lib\site-packages\sknn\backend\lasagne\mlp.py", line 241, in _initialize_impl self._create_mlp(X, w) File "C:\Anaconda\lib\site-packages\sknn\backend\lasagne\mlp.py", line 188, in _create_mlp network = self._create_layer(layer.name, layer, network) File "C:\Anaconda\lib\site-packages\sknn\backend\lasagne\mlp.py", line 171, in _create_layer network = lasagne.layers.batch_norm(network) AttributeError: 'module' object has no attribute 'batch_norm'

[alexjc]

You need the latest Lasagne from requirements.txt.

[jimmyroyer]

Thank you very much. normalize="batch" indeed works with the latest lasagne. I have a followup question however regarding get_parameters() and set_parameters(). I have the following error when I call myclassifier.get_parameters(). When I look at the structure of _mlp_to_array() I can't figure out the weights and biases. Thanks again for your help

Traceback (most recent call last): File "", line 1, in File "C:\Anaconda\lib\site-packages\sknn\nn.py", line 584, in get_parameters return [P(w, b, s.name) for s, (w, b) in zip(self.layers, self._backend._mlp_to_array())] ValueError: too many values to unpack

[alexjc]

Batch normalization seems to break the set/get because it requires many values not just bias/weight.

It's a bug ;-)

[jimmyroyer]

Thanks for your super quick responses. Can we access the final weights/biases somewhere else?

[alexjc]

Call nn._backend._mlp_to_array() directly, should be a long list of vectors.

[jimmyroyer]

Thanks again. Great package btw.