EnzymeAD / Enzyme.jl

Julia bindings for the Enzyme automatic differentiator
https://enzyme.mit.edu
MIT License
450 stars 64 forks source link

`active_reg_inner` returns `DupState` for immutable struct with untyped field #2039

Closed danielwe closed 4 hours ago

danielwe commented 4 hours ago

Consider this:

julia> struct Foo
           x
       end

julia> Enzyme.Compiler.active_reg_inner(Foo, (), nothing)
DupState::ActivityState = 2

julia> Enzyme.Compiler.active_reg_inner(Foo, (), nothing, Val(true))  # justActive
AnyState::ActivityState = 0

The way I've understood it, DupState implies that all differentiable values reachable from an object can be mutated, i.e., that there's some mutable container along the chain of references from the object to each differentiable value. But this isn't known from the type of Foo. The simplest counterexample is Foo(1.0). Then again, it could be true, like in Foo([1.0]). Wouldn't the appropriate return here be MixedState, signaling that you should be prepared for both mutable and immutable values?

wsmoses commented 4 hours ago

No because the float variable must be boxed (and thus duplicated)

On Thu, Oct 31, 2024 at 2:46 PM Daniel Wennberg @.***> wrote:

Consider this:

julia> struct Foo x end

julia> Enzyme.Compiler.active_reg_inner(Foo, (), nothing) DupState::ActivityState = 2

julia> Enzyme.Compiler.active_reg_inner(Foo, (), nothing, Val(true)) # justActive AnyState::ActivityState = 0

The way I've understood it, DupState implies that all differentiable values reachable from an object can be mutated, i.e., that there's some mutable container along the chain of references from the object to each differentiable value. But this isn't known from the type of Foo. The simplest counterexample is Foo(1.0). Then again, it could be true, like in Foo([1.0]). Wouldn't the appropriate return here be MixedState, signaling that you should be prepared for both mutable and immutable values?

— Reply to this email directly, view it on GitHub https://github.com/EnzymeAD/Enzyme.jl/issues/2039, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJTUXDO3YOUAJM4ZOXDUDDZ6KCIFAVCNFSM6AAAAABQ7AGLVWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYZDONRZGUZDEMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

wsmoses commented 4 hours ago

No because the float variable must be boxed (and thus duplicated)

danielwe commented 2 hours ago

Quick follow-up if you can spare another second, just to help my understanding:

I get that the value is held in a box, so you can mutate instances of Foo from the LLVM side or with some Ref/pointer shenanigans. But Julia seems to go to great lengths to hide this implementation detail, to the point where Foo(1.0) === Foo(1.0) is true. Doesn't this imply that mutating the boxed value would violate compiler assumptions and could wreak havoc, say if you do reverse mode autodiff with a Duplicated(Foo(0.0), Foo(0.0)) argument?

Anyway, I notice that this doesn't currently work, even with mutable struct Foo; x; end and/or putting the Foo(0.0) in a single-element vector, so perhaps the answer is simply that untyped fields aren't compatible with Enzyme?