check the swish activation

[tobigithub]

Searching for Activation Functions https://arxiv.org/abs/1710.05941

https://github.com/ChingChuan-Chen/keras_swish_beta and https://github.com/dataplayer12/swish-activation

from keras import backend as K
from keras.utils.generic_utils import get_custom_objects

class Swish(Activation):

    def __init__(self, activation, **kwargs):
        super(Swish, self).__init__(activation, **kwargs)
        self.__name__ = 'swish'

def swish(x):
    return (K.sigmoid(x) * x)

get_custom_objects().update({'swish': Swish(swish)})

[tobigithub]

Swish: http://www.wolframalpha.com/input/?i=plot+(sigmoid+x)+*+x