I have some code which calls a root finding step, via the Roots package that I'd like to be able to differentiate. It appears that whether ForwardDiff "works" in this setting depends on some subtlety that I don't quite understand.
For an MWE, consider finding the (unique) root of the cubic function f(x, k) = (x-k)^3 which has a root at x = k. Let g1(k) = find_zero(x -> f(x, k), [k-1.0,k+1.0]) and similarly let g2(k) = find_zero(x -> f(x, k), [10.0,+10.0]). Note that for values of k between -10 to +10, g1(k) == g2(k). In this range, the only difference between g1 and g2 is that the root finding step in g1 is "aware" of the value of k to look for, via the bounds I have provided, whereas it is ignorant of them in g2. For values of k in this range, the derivatives of g1 and g2 are identical, and equal to 1.0. However, I am seeing that ForwardDiff thinks the derivative of g2 is zero, while it finds the derivative of g1 to be 1.0:
I have some code which calls a root finding step, via the
Roots
package that I'd like to be able to differentiate. It appears that whetherForwardDiff
"works" in this setting depends on some subtlety that I don't quite understand.For an MWE, consider finding the (unique) root of the cubic function
f(x, k) = (x-k)^3
which has a root atx = k
. Letg1(k) = find_zero(x -> f(x, k), [k-1.0,k+1.0])
and similarly letg2(k) = find_zero(x -> f(x, k), [10.0,+10.0])
. Note that for values ofk
between -10 to +10,g1(k) == g2(k)
. In this range, the only difference betweeng1
andg2
is that the root finding step ing1
is "aware" of the value ofk
to look for, via the bounds I have provided, whereas it is ignorant of them ing2
. For values ofk
in this range, the derivatives ofg1
andg2
are identical, and equal to 1.0. However, I am seeing that ForwardDiff thinks the derivative ofg2
is zero, while it finds the derivative ofg1
to be 1.0:Julia reports that the functions are the same, and do vary with the input in the expected way:
However, the
ForwardDiff
gradients are different:Note that
FiniteDiff
works fine here (up to numerical approximation):Thanks in advance for any suggestions the
ForwardDiff
team has in understanding this difference.