Open xvlvzhu opened 5 years ago
Thanks for pointing this out! Yes, this is much cleaner and easier to understand then the old method. I will try to find time to update and test the code with this soon.
how to use? it returns two outputs, y1, y2 = flip_grad_layer(x, l) what is y1 and what is y2?
I am wondering why the second output of the "grad" function is "None" (fourth line). Can someone explain it to me?
I know this is old but it might be helpful for newcomers as well.
@Jasperty
how to use? it returns two outputs, y1, y2 = flip_grad_layer(x, l) what is y1 and what is y2?
According to tf.custom_gradient doc:
Returns: a function (x) which returns the same value as f(x)[0] and whose gradient (as calculated by tf.gradients) is determined by f(x)[1].
Simply put, y1 represents the tensor computed in the forward pass (i.e. identity) whereas y2 the downstream (custom) gradient in the backward pass.
@eliottbrion
I am wondering why the second output of the "grad" function is "None" (fourth line). Can someone explain it to me?
Referring to the same doc page as above:
In case the function takes multiple variables as input, the grad function must also return the same number of variable
This means that by providing l as required parameter to the _flip_gradlayer function the gradient will be computed for the latter variable, which is not intended and discarded by returning None.
Since tensorflow 1.7 there is a new way to redefine the gradient.