Not applying memset on reverse of jl_array_del_end

[jgreener64]

I am on Julia 1.8.5 and Enzyme main (https://github.com/EnzymeAD/Enzyme.jl/commit/16818fd3d39d0583915aee38595b54a7fcce6b58). The following fails:

using Enzyme

function f(x)
    y = filter(v -> v > 2, x)
    return sum(y)
end

x = Float64[1, 2, 3, 4]
dx = zero(x)

autodiff(Reverse, f, Active, Duplicated(x, dx))

ERROR: MethodError: no method matching jl_array_sizehint_fwd(::Ptr{LLVM.API.LLVMOpaqueBuilder}, ::Ptr{LLVM.API.LLVMOpaqueValue}, ::Ptr{Nothing}, ::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}}, ::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}}, ::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}})
Closest candidates are:
  jl_array_sizehint_fwd(::Ptr{LLVM.API.LLVMOpaqueBuilder}, ::Ptr{LLVM.API.LLVMOpaqueValue}, ::Ptr{Nothing}, ::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}}, ::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}}) at C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:5518
Stacktrace:
  [1] jl_array_sizehint_augfwd(B::Ptr{LLVM.API.LLVMOpaqueBuilder}, OrigCI::Ptr{LLVM.API.LLVMOpaqueValue}, gutils::Ptr{Nothing}, normalR::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}}, shadowR::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}}, tapeR::Ptr{Ptr{LLVM.API.LLVMOpaqueValue}})
    @ Enzyme.Compiler C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:5549
  [2] EnzymeCreatePrimalAndGradient(logic::Enzyme.Logic, todiff::LLVM.Function, retType::Enzyme.API.CDIFFE_TYPE, constant_args::Vector{Enzyme.API.CDIFFE_TYPE}, TA::Enzyme.TypeAnalysis, returnValue::Bool, dretUsed::Bool, mode::Enzyme.API.CDerivativeMode, width::Int64, additionalArg::Ptr{Nothing}, forceAnonymousTape::Bool, typeInfo::Enzyme.FnTypeInfo, uncacheable_args::Vector{Bool}, augmented::Ptr{Nothing}, atomicAdd::Bool)
    @ Enzyme.API C:\Users\Joe\.julia\dev\Enzyme\src\api.jl:128
  [3] enzyme!(job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}, mod::LLVM.Module, primalf::LLVM.Function, TT::Type, mode::Enzyme.API.CDerivativeMode, width::Int64, parallel::Bool, actualRetType::Type, wrap::Bool, modifiedBetween::Tuple{Bool, Bool}, returnPrimal::Bool, jlrules::Vector{String}, expectedTapeType::Type)
    @ Enzyme.Compiler C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:7690
  [4] codegen(output::Symbol, job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, ctx::LLVM.Context, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
    @ Enzyme.Compiler C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:9046
  [5] _thunk(job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}, ctx::Nothing, postopt::Bool)
    @ Enzyme.Compiler C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:9558
  [6] _thunk
    @ C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:9555 [inlined]
  [7] cached_compilation
    @ C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:9593 [inlined]
  [8] #s273#222
    @ C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:9651 [inlined]
  [9] var"#s273#222"(FA::Any, A::Any, TT::Any, Mode::Any, ModifiedBetween::Any, width::Any, ReturnPrimal::Any, ShadowInit::Any, World::Any, ::Any, ::Any, ::Any, ::Any, tt::Any, ::Any, ::Any, ::Any, ::Any, ::Any)
    @ Enzyme.Compiler .\none:0
 [10] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any})
    @ Core .\boot.jl:582
 [11] thunk
    @ C:\Users\Joe\.julia\dev\Enzyme\src\compiler.jl:9610 [inlined]
 [12] autodiff(#unused#::EnzymeCore.ReverseMode{false}, f::Const{typeof(f)}, #unused#::Type{Active}, args::Duplicated{Vector{Float64}})
    @ Enzyme C:\Users\Joe\.julia\dev\Enzyme\src\Enzyme.jl:199
 [13] autodiff(::EnzymeCore.ReverseMode{false}, ::typeof(f), ::Type, ::Duplicated{Vector{Float64}})
    @ Enzyme C:\Users\Joe\.julia\dev\Enzyme\src\Enzyme.jl:214
 [14] top-level scope
    @ REPL[5]:1

Version info, I get a similar error on Linux:

Julia Version 1.8.5
Commit 17cfb8e65e (2023-01-08 06:45 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 6 × Intel(R) Core(TM) i5-9400F CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 4 on 6 virtual cores
Environment:
  JULIA_NUM_THREADS = 4

[wsmoses]

this is amazing. and ironically fixed by https://github.com/EnzymeAD/Enzyme.jl/pull/626 but didn't get merged from lack of test.

@motabbara swap out the test for this and let's merge?

[wsmoses]

With that fix, we now have:

julia> autodiff(Reverse, f, Active, Duplicated(x, dx))
ERROR: Enzyme execution failed.
Not applying memset on reverse of jl_array_del_end
Stacktrace:
 [1] _deleteend!
   @ ./array.jl:1023
 [2] resize!
   @ ./array.jl:1254
 [3] filter
   @ ./array.jl:2612

Stacktrace:
 [1] throwerr(cstr::Cstring)
   @ Enzyme.Compiler ~/git/Enzyme.jl/src/compiler.jl:2910
 [2] macro expansion
   @ ~/git/Enzyme.jl/src/compiler.jl:9514 [inlined]
 [3] enzyme_call
   @ ~/git/Enzyme.jl/src/compiler.jl:9206 [inlined]
 [4] CombinedAdjointThunk
   @ ~/git/Enzyme.jl/src/compiler.jl:9169 [inlined]
 [5] autodiff(#unused#::EnzymeCore.ReverseMode{false}, f::Const{typeof(f)}, #unused#::Type{Active}, args::Duplicated{Vector{Float64}})
   @ Enzyme ~/git/Enzyme.jl/src/Enzyme.jl:205
 [6] autodiff(::EnzymeCore.ReverseMode{false}, ::typeof(f), ::Type, ::Duplicated{Vector{Float64}})
   @ Enzyme ~/git/Enzyme.jl/src/Enzyme.jl:214
 [7] top-level scope
   @ REPL[5]:1

[wsmoses]

This seems fixed on main, closing