JuliaDiff / ForwardDiff.jl

Forward Mode Automatic Differentiation for Julia
Other
892 stars 145 forks source link

Resolve `NaN` in hypot near zero #669

Open timholy opened 1 year ago

timholy commented 1 year ago

This checks whether hypot of the value component is zero, and if so switches to a next-order method.

~This may be slightly controversial since it exploits the difference between 0.0 and -0.0 to give the correct sign behavior at the origin.~ This exploits the difference between 0.0 and -0.0 to provide the same behavior exhibited for abs:

julia> ForwardDiff.derivative(abs, 0.0)
1.0

julia> ForwardDiff.derivative(abs, -0.0)
-1.0

# and now:
julia> f(x) = hypot(x, 0, 0)
f (generic function with 1 method)

julia> ForwardDiff.derivative(f, 0.0)
1.0

julia> ForwardDiff.derivative(f, -0.0)
-1.0

This implementation is consistent with the limit -> 0, e.g.,

julia> hypotvec(v) = hypot(v...)
hypotvec (generic function with 1 method)

julia> ForwardDiff.gradient(hypotvec, [0.0, -0.0, 0.0])
3-element Vector{Float64}:
  0.5773502691896258
 -0.5773502691896258
  0.5773502691896258

# for comparison:
julia> ForwardDiff.gradient(hypotvec, [1e-12, -1e-12, 1e-12])
3-element Vector{Float64}:
  0.5773502691896257
 -0.5773502691896257
  0.5773502691896257

Note this only covers the 3-arg version. For the 2-arg version I think a similar fix must be made in DiffRules.

codecov[bot] commented 1 year ago

Codecov Report

All modified lines are covered by tests :white_check_mark:

Comparison is base (d300209) 89.65% compared to head (482dde0) 87.41%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #669 +/- ## ========================================== - Coverage 89.65% 87.41% -2.25% ========================================== Files 11 10 -1 Lines 967 898 -69 ========================================== - Hits 867 785 -82 - Misses 100 113 +13 ``` | [Files](https://app.codecov.io/gh/JuliaDiff/ForwardDiff.jl/pull/669?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff) | Coverage Δ | | |---|---|---| | [src/dual.jl](https://app.codecov.io/gh/JuliaDiff/ForwardDiff.jl/pull/669?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff#diff-c3JjL2R1YWwuamw=) | `80.73% <100.00%> (-1.42%)` | :arrow_down: | ... and [9 files with indirect coverage changes](https://app.codecov.io/gh/JuliaDiff/ForwardDiff.jl/pull/669/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=JuliaDiff)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.