In the file named ''tensorlayer\activation.py'',there is a function called "sign". The code called "gradient_override_map({"sign": "QuantizeGrad"}):" which i have bolded may be a small error. It should use majuscule there, like "gradient_override_map({"Sign": "QuantizeGrad"}):",then it will work well.
@tf.RegisterGradient("QuantizeGrad")
def _sign_grad(unused_op, grad):
return tf.clip_by_value(grad, -1, 1)
def sign(x):
"""Sign function.
Clip and binarize tensor using the straight through estimator (STE) for the gradient, usually be used for
quantizing values in `Binarized Neural Networks`: https://arxiv.org/abs/1602.02830.
Parameters
----------
x : Tensor
input.
Examples
--------
>>> net = tl.layers.DenseLayer(net, 100, act=lambda x : tl.act.lrelu(x, 0.2), name='dense')
Returns
-------
Tensor
A ``Tensor`` in the same type as ``x``.
References
----------
- `Rectifier Nonlinearities Improve Neural Network Acoustic Models, Maas et al. (2013)`
http://web.stanford.edu/~awni/papers/relu_hybrid_icml2013_final.pdf
- `BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1, Courbariaux et al. (2016)`
https://arxiv.org/abs/1602.02830
"""
**with tf.get_default_graph().gradient_override_map({"sign": "QuantizeGrad"}):
return tf.sign(x, name='sign')**
In the file named ''tensorlayer\activation.py'',there is a function called "sign". The code called "gradient_override_map({"sign": "QuantizeGrad"}):" which i have bolded may be a small error. It should use majuscule there, like "gradient_override_map({"Sign": "QuantizeGrad"}):",then it will work well.
please refer to https://github.com/tensorlayer/tensorlayer/blob/master/tensorlayer/layers/activation.py