Activation Function on Elman Neural Network

[AleZonta]

I am doing some experiments with the Elman Neural Network and reading the JavaDoc I noticed this incongruence. This is what reported in the description of the class

The specified activation function will be used on all layers

and this is the description of the method setActivationFunction(ActivationFunction activation)

Set the activation function to use on each of the layers.

However, in the code, it is clearly visible that this is not true since the last layer has hard coded as activation function the value null:

public MLMethod generate() {
        BasicLayer hidden, input;

        final BasicNetwork network = new BasicNetwork();
        network.addLayer(input = new BasicLayer(this.activation, true,
                this.inputNeurons));
        network.addLayer(hidden = new BasicLayer(this.activation, true,
                this.hiddenNeurons));
        network.addLayer(new BasicLayer(null, false, this.outputNeurons));
        input.setContextFedBy(hidden);
        network.getStructure().finalizeStructure();
        network.reset();
        return network;
    }

I saw that someone else reported this in spring 2015 (without any answer) but seems like nothing changed. Is there any reason to hard code the linear activation function on the last layer?

[jeffheaton]

The activation function of an Elman network could be something other than linear, such as softmax, I will extend the class to allow that.

[jeffheaton]

Same issue as #205