search

JuliaDiff / ReverseDiff.jl

Reverse Mode Automatic Differentiation for Julia
Other
348 stars 56 forks source link

Define `typemin` for tracked reals. #225

Open manuelbb-upb opened 1 year ago

manuelbb-upb commented 1 year ago

In NNlib.maxpool we encounter typemin to initialize the prospective maximum value. typemin is not implemented for ReverseDiff.TrackedReal This definition seems to work (it allows for taking derivatives):

Base.typemin(tr::Type{<:T}) where{V, D, O, T<:ReverseDiff.TrackedReal{V, D, O}} = T(typemin(V))

but I do not really know if that is sensible.

MWE for failure:

import NNlib: maxpool
import ReverseDiff as RD
x = reshape(Float32[ 1 2; 3 4 ], (2,2,1,1))
RD.gradient(_x -> only(maxpool(_x,(2,2))), x)[:,:,1,1] # == [0 0; 0 1]