Open willtebbutt opened 3 years ago
We mostly just don't test them at all right now, because the implementation is trivial, and if it is wrong it is probably a bug in ChainRulesCore.@non_differentiable
.
Sometimes we test them if we want to be sure one is defined, or that it has the right sig.
To test that we never want to do finite differencing, so I think a test_non_differentiable
would never do finite differencing.
If that is wanted test_rrule
and test_frule
exist, and will correctly test someting non-differentiable, if it is indeed finite-difference-able
Sometimes we test them if we want to be sure one is defined, or that it has the right sig.
Agreed -- I think this + avoiding regressions is the value in such a test.
To test that we never want to do finite differencing, so I think a test_non_differentiable would never do finite differencing. If that is wanted test_rrule and test_frule exist, and will correctly test someting non-differentiable, if it is indeed finite-difference-able
Fair enough. Then I guess the functionality that we would want would always just be to test agreement with f
and that NoTangent
s are returned?
It's ocassionally not possible to test
@non_differentiable
usingtest_rrule
because finite differencing gives the wrong / unreliable results.For example, this is the case in the
Vector{Floa64}(undef, 5)
constructor implemented in https://github.com/JuliaDiff/ChainRules.jl/pull/491 -- in this case, the values of the array produced are unreliable, so finite difference estimates of the gradient aren't consistent between different calls to the function.One option would be to define a
test_non_differentiable
suite that looks something likein which we prescribe a set of standard tests. In the case that the user sets
use_finite_differencing=false
, we could just ensure thatf
does indeed returnNoTangent
s, in addition to standard things like ensuring that the primal agrees with the first output of therrule
being tested.