Swish activation function

[gigasquid]

Adding of the swish activation https://arxiv.org/abs/1710.05941 (see issue https://github.com/thinktopic/cortex/issues/242)

• Made a high level activations namespace for the activation functions that work across both cpu and cuda so that it will be easier to add more in the future.

Ran a quick test on the MNIST in the examples directory with the following:

(defn initial-description
  [input-w input-h num-classes]
  [(layers/input input-w input-h 1 :id :data)
   (layers/convolutional 5 0 1 20)
   (layers/max-pooling 2 0 2)
   (layers/relu)
   (layers/convolutional 5 0 1 50)
   (layers/max-pooling 2 0 2)
   (layers/batch-normalization)
   (layers/linear 1000)
   (layers/relu)
   (layers/linear num-classes)
   (layers/softmax :id :labels)])

relu

Classification accuracy: 0.9838

swish instead of relu

Classification accuracy: 0.984

Feedback is welcome.

[gigasquid]

😸 🎈