Easier way to add custom layers

[samlobel]

There's a lot of great functionality that comes with making a layer, like the activation, bias, and logging. I think it would be great to have an identity layer function, in the case where you want to apply activation and bias but don't want the other transformation.

For example, I'm trying to add a scaling process to convolutions that has to happen before the bias and activation. What I want to do is:

    conv = tensor.conv(dim=64, act='linear', bn=false) #don't apply bias or act yet...
    scaled_conv = tf.mul(conv, scaling_matrix)
    bias_and_act_applied = scaled_conv.sg_ident_layer() #act and bias from 'ops' applied here.
    return bias_and_act_applied

I see there's tf.identity, but it doesn't come with the nice layer add-ons.

Same as before, I'd consider making a pull request if you think it's worthwhile, but for this one I would need a little bit of direction.

Thanks again

[buriburisuri]

In that case, I created sg_bypass () layer. See sg_bypass() usage in the following codes. I'm sorry for the lack of document. I am working hard on creating a document now.

https://github.com/buriburisuri/ByteNet/blob/master/train.py https://github.com/buriburisuri/sugartensor/blob/master/sugartensor/sg_net.py

[samlobel]

Thank you! That's exactly what I was looking for.