The current implementation of leakyRelu activation function does not take the value of α or slope of activation function for X<0 into account and directly assumes it to be 0.01, which is pretty much the standard. But the option to choose a custom α(and defaulting its value to 0.01) would be more convenient for the purpose of experimentation with models.
The current implementation of leakyRelu activation function does not take the value of α or slope of activation function for X<0 into account and directly assumes it to be 0.01, which is pretty much the standard. But the option to choose a custom α(and defaulting its value to 0.01) would be more convenient for the purpose of experimentation with models.