NEActivationLayer not support PRelu?

[yanlei08]

It seems like NEActivationLayer does not support PRelu, isn't it?

    enum class ActivationFunction
    {
        LOGISTIC,        /**< Logistic ( \f$ f(x) = \frac{1}{1 + e^{-x}} \f$ ) */
        TANH,            /**< Hyperbolic tangent ( \f$ f(x) = a \cdot tanh(b \cdot x) \f$ ) */
        RELU,            /**< Rectifier ( \f$ f(x) = max(0,x) \f$ ) */
        BOUNDED_RELU,    /**< Upper Bounded Rectifier ( \f$ f(x) = min(a, max(0,x)) \f$ ) */
        LU_BOUNDED_RELU, /**< Lower and Upper Bounded Rectifier ( \f$ f(x) = min(a, max(b,x)) \f$ ) */
        LEAKY_RELU,      /**< Leaky Rectifier ( \f$ f(x)= log(1+e^x) \f$ ) */
        SOFT_RELU,       /**< Soft Rectifier ( \f$ f(x)= log(1+e^x) \f$ ) */
        ABS,             /**< Absolute ( \f$ f(x)= |x| \f$ ) */
        SQUARE,          /**< Square ( \f$ f(x)= x^2 \f$ )*/
        SQRT,            /**< Square root ( \f$ f(x) = \sqrt{x} \f$ )*/
        LINEAR           /**< Linear ( \f$ f(x)= ax + b \f$ ) */
    };

[GeorgeARM]

Hello @yanlei08,

We don't support PRelu. We have LEAKY_RELU which is essentialy the PRelu with scalar alpha. You can check the implementation here.

Is this something that you need?

[yanlei08]

Hello @GeorgeARM, Thank you for your reply. I am testing the speed of a network which contained PRelu. It seems like I need replace PRelu and retrain the network.

[GeorgeARM]

Closing this. If you need PRelu support reopen this and I will mark it as a feature request so that we can prioritize for implementation.