feat/activations: add in-place activations

[hobofan]

Activations can now be calculated in-place, requiring less memory. To use it, the same blob name should be supplied as input and output to a activation layer.

Example:

// set up linear1 layer
let linear1_cfg = LinearConfig { output_size: 1568 };
let mut lnr1_cfg = LayerConfig::new("linear1", LayerType::Linear(linear1_cfg));
lnr1_cfg.add_input("data");
lnr1_cfg.add_output("linear1_out");
net_cfg.add_layer(lnr1_cfg);
// set up sigmoid layer
let mut sigmoid_cfg = LayerConfig::new("sigmoid", LayerType::Sigmoid);
sigmoid_cfg.add_input("linear1_out"); // same input and output
sigmoid_cfg.add_output("linear1_out"); // same input and output
net_cfg.add_layer(sigmoid_cfg);

[MichaelHirn]

Yes, we should do that.

[hobofan]

commit message: "implace" -> "in-place"

[MichaelHirn]

The in-place activation layers bring the memory usage down. For the Alexnet from 3GB to 2.5GB and for the Overfeat Network from 8.2GB to 7.4GB.

The .to_string() method calls couldn't be removed, because that caused unresolvable issues with the collection String from env.

[MichaelHirn]

@homu r+

[homu]

:pushpin: Commit 223eaa4 has been approved by MichaelHirn

[homu]

:zap: Test exempted - status