search

gdalle / DifferentiationInterface.jl

An interface to various automatic differentiation backends in Julia.
https://gdalle.github.io/DifferentiationInterface.jl/DifferentiationInterface
MIT License
166 stars 13 forks source link

WaterLily using ForwardDiff and Enzyme through DifferentiationInterface #355

Closed b-fg closed 2 months ago

b-fg commented 2 months ago

Hey! I have been playing with this package to implement AD for WaterLily.jl. The ForwardDiff backend works as expected, and it provides the same result as using ForwardDiff without DI, only if compilation is performed through the appropriate Dual type. For example, the following MWE works

using WaterLily
using StaticArrays
using DifferentiationInterface
import ForwardDiff

function make_sim(θ;L=32,U=1,Re=100,mem=Array)
    function map(x,_)
        s,c = sincos(θ)
        SA[c -s; s c]*(x-SA[L,L])
    end
    function sdf(ξ,_)
        p = ξ-SA[0,clamp(ξ[1],-L/2,L/2)]
        √(p'*p)-2
    end
    Simulation((2L,2L),(U,0),L,ν=U*L/Re,body=AutoBody(sdf,map),T=typeof(θ),mem=mem)
end
function dsim(θ)
    sim = make_sim(θ)
    sim_step!(sim)
    sum(sim.flow.p)
end

function main()
    FT = Float64
    θ = FT(π/36)

    # FD
    θ1 = θ*0.999
    θ2 = θ*1.001
    println("FD value and gradient\n",
        dsim(θ),",",(dsim(θ2)-dsim(θ1))/(θ2-θ1)
    )

    # ForwardDiff
    T = typeof(ForwardDiff.Tag(dsim,FT))
    θAD = ForwardDiff.Dual{T}(θ,one(FT))
    println("\nForwardDiff value and gradient\n", dsim(θAD))

    # ForwardDiff via DifferentiationInterface
    println("\nDifferentiationInterface ForwardDiff value and gradient\n",
        value_and_gradient(dsim, AutoForwardDiff(), θ)
    )
end

main()

but commenting out the ForwardDiff lines

  # T = typeof(ForwardDiff.Tag(dsim,FT))
  # θAD = ForwardDiff.Dual{T}(θ,one(FT))
  # println("\nForwardDiff value and gradient\n", dsim(θAD))

makes the DI call to error because of the ordering of Dual tags.

Furthermore, switching to Enzyme with

value_and_gradient(dsim, AutoEnzyme(), θ)

throws an error (with a rather large stacktrace...). I am not sure how to specify forward or reverse mode when using the AutoEnzyme() backend, so it could be that by default tries to do backward mode and we have some incompatibility with that. Any help would be appreciated, thanks! :)

gdalle commented 2 months ago

I can't reproduce the ForwardDiff bug, are you perhaps using an older version of DI? Proper tag handling wasn't always present but it should work in the latest version. The two versions below have the exact same output for me, while keeping the lines you mention commented out. Incidentally, you want derivative and not gradient here, because your input is scalar.

# ForwardDiff
dsim(θ), ForwardDiff.derivative(dsim, θ)

# ForwardDiff via DI
value_and_derivative(dsim, AutoForwardDiff(), θ)
b-fg commented 2 months ago

Currently using v0.5.7

(WaterLily.jl-Examples) pkg> st DifferentiationInterface
Status `~/WaterLily.jl-Examples/Project.toml`
  [a0c0ee7d] DifferentiationInterface v0.5.7

I think that's the latest, right? So not sure what's going on. Also, thanks for that remark. Could you please clarify the practical difference in this case?

gdalle commented 2 months ago

As for the Enzyme issue, I don't think it is related to DI: I get the error too when I call Enzyme directly. Presumably something in the WaterLily Simulation object makes Enzyme mad.

```julia julia> Enzyme.autodiff(Enzyme.Forward, dsim, Enzyme.Duplicated, Enzyme.Duplicated(θ, Enzyme.make_zero(θ))) ERROR: Enzyme execution failed. Enzyme cannot deduce type Current scope: ; Function Attrs: mustprogress willreturn define "enzyme_type"="{[0]:Pointer, [8]:Pointer, [16]:Pointer, [32]:Pointer}" "enzymejl_parmtype"="138889797305424" "enzymejl_parmtype_ref"="1" [6 x {} addrspace(10)*] @preprocess_julia__make_sim_10_8465_inner.1(i64 signext "enzyme_inactive" "enzyme_type"="{[-1]:Integer}" "enzymejl_parmtype"="138891479608832" "enzymejl_parmtype_ref"="0" %0, i64 signext "enzyme_inactive" "enzyme_type"="{[-1]:Integer}" "enzymejl_parmtype"="138891479608832" "enzymejl_parmtype_ref"="0" %1, i64 signext "enzyme_inactive" "enzyme_type"="{[-1]:Integer}" "enzymejl_parmtype"="138891479608832" "enzymejl_parmtype_ref"="0" %2, {} addrspace(10)* nofree noundef nonnull readonly "enzyme_type"="{[-1]:Pointer}" "enzymejl_parmtype"="138891414058144" "enzymejl_parmtype_ref"="2" %3, double "enzyme_type"="{[-1]:Float@double}" "enzymejl_parmtype"="138891479608032" "enzymejl_parmtype_ref"="0" %4) local_unnamed_addr #11 !dbg !57 { entry: %newstruct7.i.sroa.5 = alloca [7 x i8], align 1 %newstruct7.i.sroa.5.0.sroa_idx27 = getelementptr inbounds [7 x i8], [7 x i8]* %newstruct7.i.sroa.5, i64 0, i64 0 %newstruct7.i.sroa.5.0.sroa_idx2 = getelementptr inbounds [7 x i8], [7 x i8]* %newstruct7.i.sroa.5, i64 0, i64 0 call void @llvm.lifetime.start.p0i8(i64 7, i8* %newstruct7.i.sroa.5.0.sroa_idx2) %5 = call {}*** @julia.get_pgcstack() #12, !noalias !58 %current_task1.i49 = getelementptr inbounds {}**, {}*** %5, i64 -14 %current_task1.i = bitcast {}*** %current_task1.i49 to {}** %ptls_field.i50 = getelementptr inbounds {}**, {}*** %5, i64 2 %6 = bitcast {}*** %ptls_field.i50 to i64*** %ptls_load.i5152 = load i64**, i64*** %6, align 8, !tbaa !11, !noalias !58 %7 = getelementptr inbounds i64*, i64** %ptls_load.i5152, i64 2 %safepoint.i = load i64*, i64** %7, align 8, !tbaa !15, !noalias !58 fence syncscope("singlethread") seq_cst call void @julia.safepoint(i64* %safepoint.i) #12, !dbg !61, !noalias !58 fence syncscope("singlethread") seq_cst %8 = shl i64 %0, 1, !dbg !63 %9 = mul i64 %1, %0, !dbg !63 %10 = sitofp i64 %9 to double, !dbg !65 %11 = sitofp i64 %2 to double, !dbg !65 %12 = fdiv double %10, %11, !dbg !69 %box.i = call noalias nonnull dereferenceable(8) "enzyme_type"="{[-1]:Pointer, [-1,-1]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 8, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*)) #13, !dbg !70 %13 = bitcast {} addrspace(10)* %box.i to double addrspace(10)*, !dbg !70 store double %12, double addrspace(10)* %13, align 8, !dbg !70, !tbaa !35, !alias.scope !39, !noalias !71 %box10.i = call noalias nonnull dereferenceable(48) "enzyme_type"="{[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 48, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891597143504 to {}*) to {} addrspace(10)*)) #13, !dbg !70 %14 = bitcast {} addrspace(10)* %box10.i to i8 addrspace(10)*, !dbg !70 store i8 1, i8 addrspace(10)* %14, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.5.0..sroa_raw_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 1, !dbg !70 call void @llvm.memcpy.p10i8.p0i8.i64(i8 addrspace(10)* nocapture nofree noundef writeonly align 1 dereferenceable(7) %newstruct7.i.sroa.5.0..sroa_raw_idx, i8* noundef nonnull readonly align 1 dereferenceable(7) %newstruct7.i.sroa.5.0.sroa_idx27, i64 noundef 7, i1 noundef false) #12, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.58.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 8, !dbg !70 %newstruct7.i.sroa.58.0..sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.58.0..sroa_idx to i64 addrspace(10)*, !dbg !70 store i64 %0, i64 addrspace(10)* %newstruct7.i.sroa.58.0..sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 16, !dbg !70 %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx to i64 addrspace(10)*, !dbg !70 store i64 %0, i64 addrspace(10)* %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 24, !dbg !70 %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx to double addrspace(10)*, !dbg !70 store double %4, double addrspace(10)* %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 32, !dbg !70 %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx to i64 addrspace(10)*, !dbg !70 store i64 %0, i64 addrspace(10)* %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 40, !dbg !70 %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx to double addrspace(10)*, !dbg !70 store double %4, double addrspace(10)* %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %15 = call noalias nonnull "enzyme_type"="{[-1]:Pointer, [-1,-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @jl_f_tuple, {} addrspace(10)* noundef null, {} addrspace(10)* nofree nonnull %box.i, {} addrspace(10)* nofree nonnull %box10.i, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull %3) #14, !dbg !70 %typeof.i = call "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* @julia.typeof({} addrspace(10)* nonnull %15) #15, !dbg !75 %16 = call nonnull "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @jl_f_apply_type, {} addrspace(10)* noundef null, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479609504 to {}*) to {} addrspace(10)*), {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891567787088 to {}*) to {} addrspace(10)*), {} addrspace(10)* nonnull %typeof.i) #16, !dbg !75 %box12.i = call noalias nonnull dereferenceable(8) "enzyme_type"="{[-1]:Pointer, [-1,-1]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 8, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*)) #13, !dbg !75 %17 = bitcast {} addrspace(10)* %box12.i to double addrspace(10)*, !dbg !75 store double %12, double addrspace(10)* %17, align 8, !dbg !75, !tbaa !35, !alias.scope !39, !noalias !71 %box14.i = call noalias nonnull dereferenceable(48) "enzyme_type"="{[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 48, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891597143504 to {}*) to {} addrspace(10)*)) #13, !dbg !75 %18 = bitcast {} addrspace(10)* %box14.i to i8 addrspace(10)*, !dbg !75 store i8 1, i8 addrspace(10)* %18, align 8, !dbg !75, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.5.0..sroa_raw_idx4 = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 1, !dbg !75 call void @llvm.memcpy.p10i8.p0i8.i64(i8 addrspace(10)* nocapture nofree noundef align 1 dereferenceable(7) %newstruct7.i.sroa.5.0..sroa_raw_idx4, i8* noundef nonnull align 1 dereferenceable(7) %newstruct7.i.sroa.5.0.sroa_idx27, i64 noundef 7, i1 noundef false) #12, !dbg !75, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.58.0..sroa_idx9 = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 8, !dbg !75 %newstruct7.i.sroa.58.0..sroa_cast10 = bitcast i8 addrspace(10)* %newstruct7.i.sroa.58.0..sroa_idx9 to i64 addrspace(10)*, !dbg !75 store i64 %0, i64 addrspace(10)* %newstruct7.i.sroa.58.0..sroa_cast10, align 8, !dbg !75, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 16, !dbg !75 %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx to i64 addrspace(10)*, !dbg !75 store i64 %0, i64 addrspace(10)* %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_cast, align 8, !dbg !75, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 24, !dbg !75 %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx to double addrspace(10)*, !dbg !75 store double %4, double addrspace(10)* %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_cast, align 8, !dbg !75, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 32, !dbg !75 %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx to i64 addrspace(10)*, !dbg !75 store i64 %0, i64 addrspace(10)* %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_cast, align 8, !dbg !75, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 40, !dbg !75 %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx to double addrspace(10)*, !dbg !75 store double %4, double addrspace(10)* %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_cast, align 8, !dbg !75, !tbaa !49, !alias.scope !50, !noalias !74 %19 = call noalias nonnull "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @ijl_new_structv, {} addrspace(10)* nonnull %16, {} addrspace(10)* nofree nonnull %box12.i, {} addrspace(10)* nofree nonnull %box14.i, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull %3) #14, !dbg !75 %box16.i = call noalias nonnull dereferenceable(16) "enzyme_inactive" "enzyme_type"="{[-1]:Pointer, [-1,-1]:Integer}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 16, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891407532896 to {}*) to {} addrspace(10)*)) #13, !dbg !70 %20 = bitcast {} addrspace(10)* %box16.i to i8 addrspace(10)*, !dbg !70 %newstruct3.i.sroa.0.0..sroa_cast = bitcast {} addrspace(10)* %box16.i to i64 addrspace(10)*, !dbg !70 store i64 %8, i64 addrspace(10)* %newstruct3.i.sroa.0.0..sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct3.i.sroa.4.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %20, i64 8, !dbg !70 %newstruct3.i.sroa.4.0..sroa_cast = bitcast i8 addrspace(10)* %newstruct3.i.sroa.4.0..sroa_idx to i64 addrspace(10)*, !dbg !70 store i64 %8, i64 addrspace(10)* %newstruct3.i.sroa.4.0..sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %box18.i = call noalias nonnull dereferenceable(16) "enzyme_inactive" "enzyme_type"="{[-1]:Pointer, [-1,-1]:Integer}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 16, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891407532896 to {}*) to {} addrspace(10)*)) #13, !dbg !70 %21 = bitcast {} addrspace(10)* %box18.i to i8 addrspace(10)*, !dbg !70 %newstruct4.i.sroa.0.0..sroa_cast = bitcast {} addrspace(10)* %box18.i to i64 addrspace(10)*, !dbg !70 store i64 %1, i64 addrspace(10)* %newstruct4.i.sroa.0.0..sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %newstruct4.i.sroa.4.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %21, i64 8, !dbg !70 %newstruct4.i.sroa.4.0..sroa_cast = bitcast i8 addrspace(10)* %newstruct4.i.sroa.4.0..sroa_idx to i64 addrspace(10)*, !dbg !70 store i64 0, i64 addrspace(10)* %newstruct4.i.sroa.4.0..sroa_cast, align 8, !dbg !70, !tbaa !49, !alias.scope !50, !noalias !74 %22 = call noalias nonnull "enzyme_inactive" "enzyme_type"="{[-1]:Pointer, [-1,-1]:Integer}" {} addrspace(10)* @ijl_box_int64(i64 signext %0) #17, !dbg !70, !noalias !58 %23 = call nonnull "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @ijl_apply_generic, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891468552880 to {}*) to {} addrspace(10)*), {} addrspace(10)* nonnull %19, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138889797305424 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull %box16.i, {} addrspace(10)* nofree nonnull %box18.i, {} addrspace(10)* nonnull %22) #18, !dbg !70 %24 = addrspacecast {} addrspace(10)* %23 to i8 addrspace(11)*, !dbg !70 %innersret.sroa.0.0..sroa_cast = addrspacecast {} addrspace(10)* %23 to {} addrspace(10)* addrspace(11)*, !dbg !70 %innersret.sroa.0.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.0.0..sroa_cast, align 1, !dbg !70, !noalias !76 %innersret.sroa.2.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 8, !dbg !70 %innersret.sroa.2.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.2.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !70 %innersret.sroa.2.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.2.0..sroa_cast, align 1, !dbg !70, !noalias !76 %innersret.sroa.3.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 16, !dbg !70 %innersret.sroa.3.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.3.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !70 %innersret.sroa.3.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.3.0..sroa_cast, align 1, !dbg !70, !noalias !76 %innersret.sroa.4.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 24, !dbg !70 %innersret.sroa.4.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.4.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !70 %innersret.sroa.4.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.4.0..sroa_cast, align 1, !dbg !70, !noalias !76 %innersret.sroa.5.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 32, !dbg !70 %innersret.sroa.5.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.5.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !70 %innersret.sroa.5.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.5.0..sroa_cast, align 1, !dbg !70, !noalias !76 %innersret.sroa.6.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 40, !dbg !70 %innersret.sroa.6.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.6.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !70 %innersret.sroa.6.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.6.0..sroa_cast, align 1, !dbg !70, !noalias !76 %newstruct7.i.sroa.5.0.sroa_idx3 = getelementptr inbounds [7 x i8], [7 x i8]* %newstruct7.i.sroa.5, i64 0, i64 0, !dbg !70 call void @llvm.lifetime.end.p0i8(i64 7, i8* %newstruct7.i.sroa.5.0.sroa_idx3), !dbg !70 %.fca.0.insert = insertvalue [6 x {} addrspace(10)*] poison, {} addrspace(10)* %innersret.sroa.0.0.copyload, 0, !dbg !77 %.fca.1.insert = insertvalue [6 x {} addrspace(10)*] %.fca.0.insert, {} addrspace(10)* %innersret.sroa.2.0.copyload, 1, !dbg !77 %.fca.2.insert = insertvalue [6 x {} addrspace(10)*] %.fca.1.insert, {} addrspace(10)* %innersret.sroa.3.0.copyload, 2, !dbg !77 %.fca.3.insert = insertvalue [6 x {} addrspace(10)*] %.fca.2.insert, {} addrspace(10)* %innersret.sroa.4.0.copyload, 3, !dbg !77 %.fca.4.insert = insertvalue [6 x {} addrspace(10)*] %.fca.3.insert, {} addrspace(10)* %innersret.sroa.5.0.copyload, 4, !dbg !77 %.fca.5.insert = insertvalue [6 x {} addrspace(10)*] %.fca.4.insert, {} addrspace(10)* %innersret.sroa.6.0.copyload, 5, !dbg !77 ret [6 x {} addrspace(10)*] %.fca.5.insert, !dbg !77 } Type analysis state: %14 = bitcast {} addrspace(10)* %box10.i to i8 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}, intvals: {} %innersret.sroa.2.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.2.0..sroa_cast, align 1, !dbg !34, !noalias !55: {[-1]:Pointer}, intvals: {} %.fca.0.insert = insertvalue [6 x {} addrspace(10)*] poison, {} addrspace(10)* %innersret.sroa.0.0.copyload, 0, !dbg !56: {[0]:Pointer, [8]:Anything, [9]:Anything, [10]:Anything, [11]:Anything, [12]:Anything, [13]:Anything, [14]:Anything, [15]:Anything, [16]:Anything, [17]:Anything, [18]:Anything, [19]:Anything, [20]:Anything, [21]:Anything, [22]:Anything, [23]:Anything, [24]:Anything, [25]:Anything, [26]:Anything, [27]:Anything, [28]:Anything, [29]:Anything, [30]:Anything, [31]:Anything, [32]:Anything, [33]:Anything, [34]:Anything, [35]:Anything, [36]:Anything, [37]:Anything, [38]:Anything, [39]:Anything, [40]:Anything, [41]:Anything, [42]:Anything, [43]:Anything, [44]:Anything, [45]:Anything, [46]:Anything, [47]:Anything}, intvals: {} %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 16, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double}, intvals: {} %newstruct4.i.sroa.4.0..sroa_cast = bitcast i8 addrspace(10)* %newstruct4.i.sroa.4.0..sroa_idx to i64 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %.fca.4.insert = insertvalue [6 x {} addrspace(10)*] %.fca.3.insert, {} addrspace(10)* %innersret.sroa.5.0.copyload, 4, !dbg !56: {[0]:Pointer, [8]:Pointer, [16]:Pointer, [32]:Pointer, [40]:Anything, [41]:Anything, [42]:Anything, [43]:Anything, [44]:Anything, [45]:Anything, [46]:Anything, [47]:Anything}, intvals: {} %15 = call noalias nonnull "enzyme_type"="{[-1]:Pointer, [-1,-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @jl_f_tuple, {} addrspace(10)* noundef null, {} addrspace(10)* nofree nonnull %box.i, {} addrspace(10)* nofree nonnull %box10.i, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull %3) #14, !dbg !34: {[-1]:Pointer, [-1,-1]:Pointer}, intvals: {} %box18.i = call noalias nonnull dereferenceable(16) "enzyme_inactive" "enzyme_type"="{[-1]:Pointer, [-1,-1]:Integer}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 16, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891407532896 to {}*) to {} addrspace(10)*)) #13, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %newstruct7.i.sroa.58.0..sroa_idx9 = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 8, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double, [-1,24]:Integer, [-1,25]:Integer, [-1,26]:Integer, [-1,27]:Integer, [-1,28]:Integer, [-1,29]:Integer, [-1,30]:Integer, [-1,31]:Integer, [-1,32]:Float@double}, intvals: {} %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx to i64 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double}, intvals: {} %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx to double addrspace(10)*, !dbg !52: {[-1]:Pointer, [-1,0]:Float@double}, intvals: {} %innersret.sroa.0.0..sroa_cast = addrspacecast {} addrspace(10)* %23 to {} addrspace(10)* addrspace(11)*, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer, [-1,8]:Pointer, [-1,16]:Pointer, [-1,32]:Pointer}, intvals: {} %8 = shl i64 %0, 1, !dbg !19: {[-1]:Integer}, intvals: {} %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx to double addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,0]:Float@double}, intvals: {} %9 = mul i64 %1, %0, !dbg !19: {[-1]:Integer}, intvals: {} %innersret.sroa.4.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.4.0..sroa_cast, align 1, !dbg !34, !noalias !55: {}, intvals: {} %box16.i = call noalias nonnull dereferenceable(16) "enzyme_inactive" "enzyme_type"="{[-1]:Pointer, [-1,-1]:Integer}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 16, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891407532896 to {}*) to {} addrspace(10)*)) #13, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %.fca.1.insert = insertvalue [6 x {} addrspace(10)*] %.fca.0.insert, {} addrspace(10)* %innersret.sroa.2.0.copyload, 1, !dbg !56: {[0]:Pointer, [8]:Pointer, [16]:Anything, [17]:Anything, [18]:Anything, [19]:Anything, [20]:Anything, [21]:Anything, [22]:Anything, [23]:Anything, [24]:Anything, [25]:Anything, [26]:Anything, [27]:Anything, [28]:Anything, [29]:Anything, [30]:Anything, [31]:Anything, [32]:Anything, [33]:Anything, [34]:Anything, [35]:Anything, [36]:Anything, [37]:Anything, [38]:Anything, [39]:Anything, [40]:Anything, [41]:Anything, [42]:Anything, [43]:Anything, [44]:Anything, [45]:Anything, [46]:Anything, [47]:Anything}, intvals: {} %ptls_field.i50 = getelementptr inbounds {}**, {}*** %5, i64 2: {[-1]:Pointer, [-1,0]:Pointer}, intvals: {} %newstruct3.i.sroa.4.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %20, i64 8, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %newstruct7.i.sroa.5.0..sroa_raw_idx4 = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 1, !dbg !52: {[-1]:Pointer, [-1,7]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Float@double, [-1,31]:Integer, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Float@double}, intvals: {} %newstruct7.i.sroa.58.0..sroa_cast10 = bitcast i8 addrspace(10)* %newstruct7.i.sroa.58.0..sroa_idx9 to i64 addrspace(10)*, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double, [-1,24]:Integer, [-1,25]:Integer, [-1,26]:Integer, [-1,27]:Integer, [-1,28]:Integer, [-1,29]:Integer, [-1,30]:Integer, [-1,31]:Integer, [-1,32]:Float@double}, intvals: {} i64 %0: {[-1]:Integer}, intvals: {} i64 %1: {[-1]:Integer}, intvals: {} i64 %2: {[-1]:Integer}, intvals: {} {} addrspace(10)* %3: {[-1]:Pointer}, intvals: {} double %4: {[-1]:Float@double}, intvals: {} %box10.i = call noalias nonnull dereferenceable(48) "enzyme_type"="{[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 48, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891597143504 to {}*) to {} addrspace(10)*)) #13, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}, intvals: {} %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 32, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double}, intvals: {} %box12.i = call noalias nonnull dereferenceable(8) "enzyme_type"="{[-1]:Pointer, [-1,-1]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 8, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*)) #13, !dbg !52: {[-1]:Pointer, [-1,-1]:Float@double}, intvals: {} {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138889797305424 to {}*) to {} addrspace(10)*): {[-1]:Anything}, intvals: {} %.fca.5.insert = insertvalue [6 x {} addrspace(10)*] %.fca.4.insert, {} addrspace(10)* %innersret.sroa.6.0.copyload, 5, !dbg !56: {[0]:Pointer, [8]:Pointer, [16]:Pointer, [32]:Pointer}, intvals: {} %12 = fdiv double %10, %11, !dbg !32: {[-1]:Float@double}, intvals: {} %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx to double addrspace(10)*, !dbg !52: {[-1]:Pointer, [-1,0]:Float@double, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double}, intvals: {} {}* inttoptr (i64 138891468552880 to {}*): {[-1]:Anything}, intvals: {} [6 x {} addrspace(10)*] poison: {[-1]:Anything}, intvals: {} i64 1: {[-1]:Integer}, intvals: {1,} %18 = bitcast {} addrspace(10)* %box14.i to i8 addrspace(10)*, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}, intvals: {} {}* inttoptr (i64 138891597143504 to {}*): {[-1]:Anything}, intvals: {} i64 0: {[-1]:Anything}, intvals: {0,} %innersret.sroa.3.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 16, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer, [-1,16]:Pointer}, intvals: {} %innersret.sroa.0.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.0.0..sroa_cast, align 1, !dbg !34, !noalias !55: {[-1]:Pointer}, intvals: {} %newstruct7.i.sroa.5.0.sroa_idx2 = getelementptr inbounds [7 x i8], [7 x i8]* %newstruct7.i.sroa.5, i64 0, i64 0: {[-1]:Pointer}, intvals: {} %innersret.sroa.4.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.4.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !34: {[-1]:Pointer, [-1,8]:Pointer}, intvals: {} %newstruct7.i.sroa.5.0.sroa_idx27 = getelementptr inbounds [7 x i8], [7 x i8]* %newstruct7.i.sroa.5, i64 0, i64 0: {[-1]:Pointer}, intvals: {} {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891468552880 to {}*) to {} addrspace(10)*): {[-1]:Anything}, intvals: {} %typeof.i = call "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* @julia.typeof({} addrspace(10)* nonnull %15) #15, !dbg !52: {[-1]:Pointer}, intvals: {} {}* inttoptr (i64 138891479608032 to {}*): {[-1]:Anything}, intvals: {} %newstruct7.i.sroa.5 = alloca [7 x i8], align 1: {[-1]:Pointer}, intvals: {} %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx to i64 addrspace(10)*, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double}, intvals: {} %innersret.sroa.5.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.5.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer}, intvals: {} %safepoint.i = load i64*, i64** %7, align 8, !tbaa !15, !noalias !8: {}, intvals: {} %innersret.sroa.2.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 8, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer, [-1,8]:Pointer, [-1,24]:Pointer}, intvals: {} %22 = call noalias nonnull "enzyme_inactive" "enzyme_type"="{[-1]:Pointer, [-1,-1]:Integer}" {} addrspace(10)* @ijl_box_int64(i64 signext %0) #17, !dbg !34, !noalias !8: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479609504 to {}*) to {} addrspace(10)*): {[-1]:Anything}, intvals: {} %innersret.sroa.5.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.5.0..sroa_cast, align 1, !dbg !34, !noalias !55: {[-1]:Pointer}, intvals: {} {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891567787088 to {}*) to {} addrspace(10)*): {[-1]:Anything}, intvals: {} %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 24, !dbg !52: {[-1]:Pointer, [-1,0]:Float@double, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double}, intvals: {} {}* inttoptr (i64 138891567787088 to {}*): {[-1]:Anything}, intvals: {} %box14.i = call noalias nonnull dereferenceable(48) "enzyme_type"="{[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 48, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891597143504 to {}*) to {} addrspace(10)*)) #13, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Float@double}, intvals: {} %innersret.sroa.6.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 40, !dbg !34: {[-1]:Pointer}, intvals: {} %box.i = call noalias nonnull dereferenceable(8) "enzyme_type"="{[-1]:Pointer, [-1,-1]:Float@double}" {} addrspace(10)* @julia.gc_alloc_obj({}** nonnull %current_task1.i, i64 noundef 8, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*)) #13, !dbg !34: {[-1]:Pointer, [-1,-1]:Float@double}, intvals: {} %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx14.sroa_idx to i64 addrspace(10)*, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double}, intvals: {} %10 = sitofp i64 %9 to double, !dbg !23: {[-1]:Float@double}, intvals: {} {}* inttoptr (i64 138891407532896 to {}*): {[-1]:Anything}, intvals: {} %16 = call nonnull "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @jl_f_apply_type, {} addrspace(10)* noundef null, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479609504 to {}*) to {} addrspace(10)*), {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891567787088 to {}*) to {} addrspace(10)*), {} addrspace(10)* nonnull %typeof.i) #16, !dbg !52: {[-1]:Pointer}, intvals: {} %innersret.sroa.3.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.3.0..sroa_cast, align 1, !dbg !34, !noalias !55: {[-1]:Pointer}, intvals: {} %newstruct7.i.sroa.6.sroa.0.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 16, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Float@double}, intvals: {} %11 = sitofp i64 %2 to double, !dbg !23: {[-1]:Float@double}, intvals: {} %newstruct7.i.sroa.5.0.sroa_idx3 = getelementptr inbounds [7 x i8], [7 x i8]* %newstruct7.i.sroa.5, i64 0, i64 0, !dbg !34: {[-1]:Pointer}, intvals: {} %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 24, !dbg !34: {[-1]:Pointer, [-1,0]:Float@double, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double}, intvals: {} %innersret.sroa.4.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 24, !dbg !34: {[-1]:Pointer, [-1,8]:Pointer}, intvals: {} %newstruct7.i.sroa.58.0..sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.58.0..sroa_idx to i64 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double, [-1,24]:Integer, [-1,25]:Integer, [-1,26]:Integer, [-1,27]:Integer, [-1,28]:Integer, [-1,29]:Integer, [-1,30]:Integer, [-1,31]:Integer, [-1,32]:Float@double}, intvals: {} {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891597143504 to {}*) to {} addrspace(10)*): {[-1]:Anything}, intvals: {} %ptls_load.i5152 = load i64**, i64*** %6, align 8, !tbaa !11, !noalias !8: {[-1]:Pointer}, intvals: {} %23 = call nonnull "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @ijl_apply_generic, {} addrspace(10)* noundef addrspacecast ({}* inttoptr (i64 138891468552880 to {}*) to {} addrspace(10)*), {} addrspace(10)* nonnull %19, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138889797305424 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull %box16.i, {} addrspace(10)* nofree nonnull %box18.i, {} addrspace(10)* nonnull %22) #18, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer, [-1,8]:Pointer, [-1,16]:Pointer, [-1,32]:Pointer}, intvals: {} %6 = bitcast {}*** %ptls_field.i50 to i64***: {[-1]:Pointer, [-1,0]:Pointer}, intvals: {} %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 40, !dbg !34: {[-1]:Pointer, [-1,0]:Float@double}, intvals: {} %19 = call noalias nonnull "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @ijl_new_structv, {} addrspace(10)* nonnull %16, {} addrspace(10)* nofree nonnull %box12.i, {} addrspace(10)* nofree nonnull %box14.i, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull %3) #14, !dbg !52: {[-1]:Pointer}, intvals: {} %newstruct7.i.sroa.5.0..sroa_raw_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 1, !dbg !34: {[-1]:Pointer, [-1,7]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Float@double, [-1,31]:Integer, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Float@double}, intvals: {} %7 = getelementptr inbounds i64*, i64** %ptls_load.i5152, i64 2: {[-1]:Pointer}, intvals: {} %newstruct4.i.sroa.0.0..sroa_cast = bitcast {} addrspace(10)* %box18.i to i64 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.6.sroa.5.0.newstruct7.i.sroa.6.0..sroa_raw_idx.sroa_idx to double addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,0]:Float@double, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double}, intvals: {} %newstruct4.i.sroa.4.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %21, i64 8, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %newstruct7.i.sroa.7.sroa.5.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 40, !dbg !52: {[-1]:Pointer, [-1,0]:Float@double}, intvals: {} %20 = bitcast {} addrspace(10)* %box16.i to i8 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891407532896 to {}*) to {} addrspace(10)*): {[-1]:Anything}, intvals: {} {}* inttoptr (i64 138889797305424 to {}*): {[-1]:Anything}, intvals: {} %newstruct3.i.sroa.0.0..sroa_cast = bitcast {} addrspace(10)* %box16.i to i64 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %innersret.sroa.6.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.6.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !34: {[-1]:Pointer}, intvals: {} {} addrspace(10)* addrspacecast ({}* inttoptr (i64 138891479608032 to {}*) to {} addrspace(10)*): {[-1]:Anything}, intvals: {} %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_cast = bitcast i8 addrspace(10)* %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx.sroa_idx to i64 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double}, intvals: {} %21 = bitcast {} addrspace(10)* %box18.i to i8 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} {}* inttoptr (i64 138891479609504 to {}*): {[-1]:Anything}, intvals: {} %current_task1.i = bitcast {}*** %current_task1.i49 to {}**: {[-1]:Pointer}, intvals: {} %13 = bitcast {} addrspace(10)* %box.i to double addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,-1]:Float@double}, intvals: {} %innersret.sroa.2.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.2.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer, [-1,8]:Pointer, [-1,24]:Pointer}, intvals: {} %current_task1.i49 = getelementptr inbounds {}**, {}*** %5, i64 -14: {[-1]:Pointer}, intvals: {} %newstruct7.i.sroa.7.sroa.0.0.newstruct7.i.sroa.7.0..sroa_raw_idx19.sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %18, i64 32, !dbg !52: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Float@double}, intvals: {} %newstruct7.i.sroa.58.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(10)* %14, i64 8, !dbg !34: {[-1]:Pointer, [-1,0]:Integer, [-1,1]:Integer, [-1,2]:Integer, [-1,3]:Integer, [-1,4]:Integer, [-1,5]:Integer, [-1,6]:Integer, [-1,7]:Integer, [-1,8]:Integer, [-1,9]:Integer, [-1,10]:Integer, [-1,11]:Integer, [-1,12]:Integer, [-1,13]:Integer, [-1,14]:Integer, [-1,15]:Integer, [-1,16]:Float@double, [-1,24]:Integer, [-1,25]:Integer, [-1,26]:Integer, [-1,27]:Integer, [-1,28]:Integer, [-1,29]:Integer, [-1,30]:Integer, [-1,31]:Integer, [-1,32]:Float@double}, intvals: {} %.fca.3.insert = insertvalue [6 x {} addrspace(10)*] %.fca.2.insert, {} addrspace(10)* %innersret.sroa.4.0.copyload, 3, !dbg !56: {[0]:Pointer, [8]:Pointer, [16]:Pointer, [32]:Anything, [33]:Anything, [34]:Anything, [35]:Anything, [36]:Anything, [37]:Anything, [38]:Anything, [39]:Anything, [40]:Anything, [41]:Anything, [42]:Anything, [43]:Anything, [44]:Anything, [45]:Anything, [46]:Anything, [47]:Anything}, intvals: {} %17 = bitcast {} addrspace(10)* %box12.i to double addrspace(10)*, !dbg !52: {[-1]:Pointer, [-1,-1]:Float@double}, intvals: {} %5 = call {}*** @julia.get_pgcstack() #12, !noalias !8: {[-1]:Pointer, [-1,16]:Pointer}, intvals: {} %innersret.sroa.3.0..sroa_cast = bitcast i8 addrspace(11)* %innersret.sroa.3.0..sroa_idx to {} addrspace(10)* addrspace(11)*, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer, [-1,16]:Pointer}, intvals: {} %newstruct3.i.sroa.4.0..sroa_cast = bitcast i8 addrspace(10)* %newstruct3.i.sroa.4.0..sroa_idx to i64 addrspace(10)*, !dbg !34: {[-1]:Pointer, [-1,-1]:Integer}, intvals: {} %.fca.2.insert = insertvalue [6 x {} addrspace(10)*] %.fca.1.insert, {} addrspace(10)* %innersret.sroa.3.0.copyload, 2, !dbg !56: {[0]:Pointer, [8]:Pointer, [16]:Pointer, [24]:Anything, [25]:Anything, [26]:Anything, [27]:Anything, [28]:Anything, [29]:Anything, [30]:Anything, [31]:Anything, [32]:Anything, [33]:Anything, [34]:Anything, [35]:Anything, [36]:Anything, [37]:Anything, [38]:Anything, [39]:Anything, [40]:Anything, [41]:Anything, [42]:Anything, [43]:Anything, [44]:Anything, [45]:Anything, [46]:Anything, [47]:Anything}, intvals: {} %24 = addrspacecast {} addrspace(10)* %23 to i8 addrspace(11)*, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer, [-1,8]:Pointer, [-1,16]:Pointer, [-1,32]:Pointer}, intvals: {} %innersret.sroa.5.0..sroa_idx = getelementptr inbounds i8, i8 addrspace(11)* %24, i64 32, !dbg !34: {[-1]:Pointer, [-1,0]:Pointer}, intvals: {} %innersret.sroa.6.0.copyload = load {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %innersret.sroa.6.0..sroa_cast, align 1, !dbg !34, !noalias !55: {}, intvals: {} Cannot deduce type of copy call void @llvm.memcpy.p10i8.p0i8.i64(i8 addrspace(10)* nocapture nofree noundef writeonly align 1 dereferenceable(7) %newstruct7.i.sroa.5.0..sroa_raw_idx, i8* noundef nonnull readonly align 1 dereferenceable(7) %newstruct7.i.sroa.5.0.sroa_idx27, i64 noundef 7, i1 noundef false) #12, !dbg !34, !tbaa !49, !alias.scope !50, !noalias !51 Caused by: Stacktrace: [1] #make_sim#10 @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:15 [2] #make_sim#10 @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:0 within MethodInstance for var"#make_sim#10"(::Int64, ::Int64, ::Int64, ::Type{Array}, ::typeof(make_sim), ::Float64) Stacktrace: [1] throwerr(cstr::Cstring) @ Enzyme.Compiler ~/.julia/packages/Enzyme/Pljwm/src/compiler.jl:1678 [2] #make_sim#10 @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:15 [inlined] [3] fwddiffejulia__make_sim_10_8465_inner_1wrap @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:0 [4] macro expansion @ ~/.julia/packages/Enzyme/Pljwm/src/compiler.jl:6630 [inlined] [5] enzyme_call(::Val{…}, ::Ptr{…}, ::Type{…}, ::Val{…}, ::Val{…}, ::Type{…}, ::Type{…}, ::EnzymeCore.Const{…}, ::Type{…}, ::EnzymeCore.Const{…}, ::EnzymeCore.Const{…}, ::EnzymeCore.Const{…}, ::EnzymeCore.Const{…}, ::EnzymeCore.Const{…}, ::EnzymeCore.Duplicated{…}) @ Enzyme.Compiler ~/.julia/packages/Enzyme/Pljwm/src/compiler.jl:6231 [6] (::Enzyme.Compiler.ForwardModeThunk{…})(::EnzymeCore.Const{…}, ::EnzymeCore.Const{…}, ::Vararg{…}) @ Enzyme.Compiler ~/.julia/packages/Enzyme/Pljwm/src/compiler.jl:6111 [7] runtime_generic_fwd(activity::Type{…}, width::Val{…}, RT::Val{…}, f::var"##make_sim#10", df::Nothing, primal_1::Int64, shadow_1_1::Nothing, primal_2::Int64, shadow_2_1::Nothing, primal_3::Int64, shadow_3_1::Nothing, primal_4::Type{…}, shadow_4_1::Nothing, primal_5::typeof(make_sim), shadow_5_1::Nothing, primal_6::Float64, shadow_6_1::Float64) @ Enzyme.Compiler ~/.julia/packages/Enzyme/Pljwm/src/rules/jitrules.jl:210 [8] make_sim @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:6 [inlined] [9] dsim @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:19 [inlined] [10] fwddiffejulia_dsim_8822wrap @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:0 [11] macro expansion @ ~/.julia/packages/Enzyme/Pljwm/src/compiler.jl:6630 [inlined] [12] enzyme_call @ ~/.julia/packages/Enzyme/Pljwm/src/compiler.jl:6231 [inlined] [13] ForwardModeThunk @ ~/.julia/packages/Enzyme/Pljwm/src/compiler.jl:6111 [inlined] [14] autodiff @ ~/.julia/packages/Enzyme/Pljwm/src/Enzyme.jl:427 [inlined] [15] autodiff(mode::EnzymeCore.ForwardMode{…}, f::typeof(dsim), ::Type{…}, args::EnzymeCore.Duplicated{…}) @ Enzyme ~/.julia/packages/Enzyme/Pljwm/src/Enzyme.jl:326 [16] top-level scope @ ~/Work/GitHub/Julia/Sandbox/waterlily.jl:33 Some type information was truncated. Use `show(err)` to see complete types. ```

The AutoEnzyme() backend object picks the best mode based on the operator you're applying. In the case of derivative, it will be forward mode, while in the case of gradient, it will be reverse mode. You can force a given mode with a keyword argument:

help?> AutoEnzyme
search: AutoEnzyme

  AutoEnzyme{M}

  Struct used to select the Enzyme.jl (https://github.com/EnzymeAD/Enzyme.jl) backend for automatic differentiation.

  Defined by ADTypes.jl (https://github.com/SciML/ADTypes.jl).

  Constructors
  ≡≡≡≡≡≡≡≡≡≡≡≡

  AutoEnzyme(; mode=nothing)

  Fields
  ≡≡≡≡≡≡

    •  mode::M: can be either
       • an object subtyping EnzymeCore.Mode (like EnzymeCore.Forward or EnzymeCore.Reverse) if a specific mode is required
       • nothing to choose the best mode automatically
b-fg commented 2 months ago

Thanks for the info! And yes, that's the error I get with enzyme. It points to a memcpy when creating the Simulation struct and says that "Enzyme cannot deduce type", but I cannot see information of where this is exactly happening.

gdalle commented 2 months ago

I think that's the latest, right? So not sure what's going on.

Can you run the full code below in a fresh Julia REPL (the initial environment doesn't matter) and see if it still errors?

```julia using Pkg Pkg.activate(temp=true) Pkg.add(["WaterLily", "StaticArrays", "DifferentiationInterface", "ForwardDiff"]) using WaterLily using StaticArrays using DifferentiationInterface import ForwardDiff function make_sim(θ; L=32, U=1, Re=100, mem=Array) function map(x, _) s, c = sincos(θ) SA[c -s; s c] * (x - SA[L, L]) end function sdf(ξ, _) p = ξ - SA[0, clamp(ξ[1], -L / 2, L / 2)] √(p' * p) - 2 end Simulation((2L, 2L), (U, 0), L, ν=U * L / Re, body=AutoBody(sdf, map), T=typeof(θ), mem=mem) end function dsim(θ) sim = make_sim(θ) sim_step!(sim) sum(sim.flow.p) end θ = Float64(π / 36) # ForwardDiff dsim(θ), ForwardDiff.derivative(dsim, θ) # ForwardDiff via DI value_and_derivative(dsim, AutoForwardDiff(), θ) ```

Also, thanks for that remark. Could you please clarify the practical difference in this case?

As for the operators:

For scalar input and output, both should work, but I didn't make sure that gradient does since this case is covered by derivative already and it will be more efficient

gdalle commented 2 months ago

Thanks for the info! And yes, that's the error I get with enzyme. It points to a memcpy when creating the Simulation struct and says that "Enzyme cannot deduce type", but I cannot see information of where this is exactly happening.

I would open an issue on the Enzyme.jl repo with my MWE (not using DifferentiationInterface at all) to see what @wsmoses thinks.

b-fg commented 2 months ago

The code you shared does not error indeed. But note that it contains ForwardDiff.derivative(dsim, θ), which I assume it forces compilation of dsim with the appropriate ForwardDiff.Dual type. If this line is removed, this code produces an error for me

```julia using Pkg Pkg.activate(temp=true) Pkg.add(["WaterLily", "StaticArrays", "DifferentiationInterface", "ForwardDiff"]) using WaterLily using StaticArrays using DifferentiationInterface import ForwardDiff function make_sim(θ; L=32, U=1, Re=100, mem=Array) function map(x, _) s, c = sincos(θ) SA[c -s; s c] * (x - SA[L, L]) end function sdf(ξ, _) p = ξ - SA[0, clamp(ξ[1], -L / 2, L / 2)] √(p' * p) - 2 end Simulation((2L, 2L), (U, 0), L, ν=U * L / Re, body=AutoBody(sdf, map), T=typeof(θ), mem=mem) end function dsim(θ) sim = make_sim(θ) sim_step!(sim) sum(sim.flow.p) end θ = Float64(π / 36) # ForwardDiff via DI value_and_derivative(dsim, AutoForwardDiff(), θ) ```

And noted about Enzyme, I will open an issue there. Thanks!

gdalle commented 2 months ago

Okay now I got the error. It is very weird because the first call to ForwardDiff.derivative should be completely independent of the second one to DI.derivative. Investigating

gdalle commented 2 months ago

Is there a version of the code without the KernelAbstractions? I'm not familiar with that library and its macros may mess with my understanding of what goes on inside WaterLily.measure

b-fg commented 2 months ago

Oh, note that we use ForwardDiff within the code:

https://github.com/WaterLily-jl/WaterLily.jl/blob/68b7c6b5658e6b3ca06b69ffa35fcdd5aabbebee/src/AutoBody.jl#L129-L130

So I think that the tags are getting mixed up there. That's why we sometimes create the Dual type with the appropriate tag, eg

T = typeof(ForwardDiff.Tag(dsim, Float64))
θ = ForwardDiff.Dual{T}(θ, one(Float64))
gdalle commented 2 months ago

Actually, it seems KernelAbstractions may also mess with the tagging of the arguments, see for example:

b-fg commented 2 months ago

To get KA out of the loop, just run with julia -t 1 and WaterLily defaults to non-KA kernels.

gdalle commented 2 months ago

Okay, we now have a simpler stacktrace, but the error is still here. Weirdly enough, in this order both calls succeed

ForwardDiff.derivative(dsim, θ)
derivative(dsim, AutoForwardDiff(), θ)

but in this order both calls error

derivative(dsim, AutoForwardDiff(), θ)
ForwardDiff.derivative(dsim, θ)

That's beyond superweird. Is there any global state in WaterLily that might be altered?

b-fg commented 2 months ago

Yes, that is weird indeed :/ If you could please clarify what do you mean by global state?

gdalle commented 2 months ago

Maybe a global variable of some form that the first call could modify. I think you're right and it has to do with compilation though. I have removed KA, removed the @loop macro in measure!, still getting the same bug.

gdalle commented 2 months ago

I think I know what is happening. If I'm right, the global state I'm looking for is ForwardDiff.TAGCOUNT. I was creating tags in a way that doesn't increment it properly. Let my try a quick fix, by creating tags with Tag(f, eltype(x)) (which increments the counter) instead of Tag{typeof(f), eltype(x)} (which doesn't).

b-fg commented 2 months ago

There is no such thing AFAIK Sure, thanks!. The only thing is that we use ForwardDiff within measure, and I have been doing some checks and that is indeed the problem. When I don't use the functions in there, ie ForwardDiff.jacobian, ForwardDiff.derivative, ForwardDiff.gradient the error does not show up.

gdalle commented 2 months ago

Can you try it with the branch from https://github.com/gdalle/DifferentiationInterface.jl/pull/357?

b-fg commented 2 months ago

Works as intended! Thank you very much for the quick fix. And I will open an issue in Enzyme.jl regarding the other stuff. Cheers :)