leaky relu slope, parameter?

[aosokin]

Hi, would it be possible to convert the slope of the negative piece of leaky relu to a parameter instead of hard-coded value?

[buriburisuri]

@aosokin

You can do it as follows:

@tf.sg_sugar_func
def my_relu(x, opt):
   return tf.select(tf.greater(x, 0), x, opt.slope * x)

tf.sg_inject_func(my_relu)

.
.
.

y = x.sg_conv().my_relu(slope=0.01)

# or

with tf.sg_scope(slope=0.01)
  y = x.sg_conv().my_relu() 

.
.

Thanks

[aosokin]

Thanks, this works! I suggest adding leaky ReLU slope as a parameter to the main code.