search

EnzymeAD / Enzyme.jl

Julia bindings for the Enzyme automatic differentiator
https://enzyme.mit.edu
MIT License
455 stars 64 forks source link

DI tests failing on 1.11: how to fix? #1951

Closed gdalle closed 1 month ago

gdalle commented 1 month ago

I know that there are still a few compatibility issues with 1.11 (#1358) but I'm wondering if changes are necessary on the DI side. My test suite which used to pass on 1.10 suddenly segfaults. Here is the CI log, and here is an MWE:

using Pkg
Pkg.activate(; temp=true)
Pkg.add("Enzyme")
Pkg.add(;
    url="https://github.com/gdalle/DifferentiationInterface.jl",
    subdir="DifferentiationInterface",
)
Pkg.add(;
    url="https://github.com/gdalle/DifferentiationInterface.jl",
    subdir="DifferentiationInterfaceTest",
)

using DifferentiationInterface
using DifferentiationInterfaceTest
using Enzyme

test_differentiation(AutoEnzyme(); logging=true)

Stacktrace (part of it): stacktrace.txt

Key bits:

Got exception outside of a @test
  Enzyme compilation failed.
  Current scope:   in
  ; Function Attrs: mustprogress willreturn
  define "enzyme_type"="{[-1]:Float@double}" double @preprocess_julia_arr_to_num_linalg_69316({} addrspace(10)* nocapture noundef nonnull readonly align 8 dereferenceable(32) "enzyme_type"="{[-1]:Pointer, [-1,0]:Pointer, [-1,8]:Pointer, [-1,8,0]:Integer, [-1,8,1]:Integer, [-1,8,2]:Integer, [-1,8,3]:Integer, [-1,8,4]:Integer, [-1,8,5]:Integer, [-1,8,6]:Integer, [-1,8,7]:Integer, [-1,8,8]:Pointer, [-1,8,8,-1]:Float@double, [-1,16]:Integer, [-1,17]:Integer, [-1,18]:Integer, [-1,19]:Integer, [-1,20]:Integer, [-1,21]:Integer, [-1,22]:Integer, [-1,23]:Integer, [-1,24]:Integer, [-1,25]:Integer, [-1,26]:Integer, [-1,27]:Integer, [-1,28]:Integer, [-1,29]:Integer, [-1,30]:Integer, [-1,31]:Integer}" "enzymejl_parmtype"="4790499984" "enzymejl_parmtype_ref"="2" %0) local_unnamed_addr #24 !dbg !915 {
  top:

...

Did not have return index set when differentiating function
   call  %50 = call nonnull "enzyme_type"="{[-1]:Pointer}" {} addrspace(10)* %49({} addrspace(10)* %36, i64 %48, i64 %39) #29, !dbg !134
   augmentcall  %_augmented98 = call { i8*, {} addrspace(10)* } %72({} addrspace(10)* %55, {} addrspace(10)* %"'ipl80", i64 %67, i64 %"'ipc96", i64 %58, i64 %58), !dbg !174

  Stacktrace:
    [1] copy
      @ ./array.jl:350
    [2] unaliascopy
      @ ./abstractarray.jl:1516
    [3] unalias
      @ ./abstractarray.jl:1500
    [4] broadcast_unalias
      @ ./broadcast.jl:941
    [5] preprocess
      @ ./broadcast.jl:948
    [6] preprocess_args
      @ ./broadcast.jl:950
    [7] preprocess
      @ ./broadcast.jl:947
    [8] copyto!
      @ ./broadcast.jl:964
    [9] copyto!
      @ ./broadcast.jl:920
   [10] copy
      @ ./broadcast.jl:892
   [11] materialize
      @ ./broadcast.jl:867
   [12] arr_to_num_linalg
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/scenarios/default.jl:173

  Stacktrace:
    [1] julia_error(cstr::Cstring, val::Ptr{LLVM.API.LLVMOpaqueValue}, errtype::Enzyme.API.ErrorType, data::Ptr{Nothing}, data2::Ptr{LLVM.API.LLVMOpaqueValue}, B::Ptr{LLVM.API.LLVMOpaqueBuilder})
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:2294
    [2] EnzymeCreatePrimalAndGradient(logic::Enzyme.Logic, todiff::LLVM.Function, retType::Enzyme.API.CDIFFE_TYPE, constant_args::Vector{Enzyme.API.CDIFFE_TYPE}, TA::Enzyme.TypeAnalysis, returnValue::Bool, dretUsed::Bool, mode::Enzyme.API.CDerivativeMode, runtimeActivity::Bool, width::Int64, additionalArg::Ptr{Nothing}, forceAnonymousTape::Bool, typeInfo::Enzyme.FnTypeInfo, uncacheable_args::Vector{Bool}, augmented::Ptr{Nothing}, atomicAdd::Bool)
      @ Enzyme.API ~/.julia/packages/Enzyme/Vjlrr/src/api.jl:253
    [3] enzyme!(job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}, mod::LLVM.Module, primalf::LLVM.Function, TT::Type, mode::Enzyme.API.CDerivativeMode, width::Int64, parallel::Bool, actualRetType::Type, wrap::Bool, modifiedBetween::Tuple{Bool, Bool}, returnPrimal::Bool, expectedTapeType::Type, loweredArgs::Set{Int64}, boxedArgs::Set{Int64})
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:4706
    [4] codegen(output::Symbol, job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, toplevel::Bool, strip::Bool, validate::Bool, only_entry::Bool, parent_job::GPUCompiler.CompilerJob{GPUCompiler.NativeCompilerTarget, Enzyme.Compiler.PrimalCompilerParams})
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:7801
    [5] codegen
      @ ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:6638 [inlined]
    [6] (::GPUCompiler.var"#187#198"{GPUCompiler.CompilerJob{GPUCompiler.NativeCompilerTarget, Enzyme.Compiler.PrimalCompilerParams}, GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}})()
      @ GPUCompiler ~/.julia/packages/GPUCompiler/2CW9L/src/driver.jl:224
    [7] get!(default::GPUCompiler.var"#187#198"{GPUCompiler.CompilerJob{GPUCompiler.NativeCompilerTarget, Enzyme.Compiler.PrimalCompilerParams}, GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}}, h::Dict{GPUCompiler.CompilerJob, String}, key::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams})
      @ Base ./dict.jl:458
    [8] macro expansion
      @ ~/.julia/packages/GPUCompiler/2CW9L/src/driver.jl:223 [inlined]
    [9] emit_llvm(job::GPUCompiler.CompilerJob; toplevel::Bool, libraries::Bool, optimize::Bool, cleanup::Bool, validate::Bool, only_entry::Bool)
      @ GPUCompiler ~/.julia/packages/GPUCompiler/2CW9L/src/utils.jl:108
   [10] codegen(output::Symbol, job::GPUCompiler.CompilerJob; toplevel::Bool, libraries::Bool, optimize::Bool, cleanup::Bool, validate::Bool, strip::Bool, only_entry::Bool, parent_job::Nothing)
      @ GPUCompiler ~/.julia/packages/GPUCompiler/2CW9L/src/driver.jl:100
   [11] codegen
      @ ~/.julia/packages/GPUCompiler/2CW9L/src/driver.jl:82 [inlined]
   [12] codegen(output::Symbol, job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, toplevel::Bool, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:6694
   [13] codegen
      @ ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:6638 [inlined]
   [14] _thunk(job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}, postopt::Bool)
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:8909
   [15] _thunk
      @ ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:8909 [inlined]
   [16] cached_compilation
      @ ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:8950 [inlined]
   [17] thunkbase(ctx::LLVM.Context, mi::Core.MethodInstance, ::Val{0x0000000000006854}, ::Type{Const{DifferentiationInterface.var"#inner_gradient#46"{typeof(DifferentiationInterfaceTest.arr_to_num_linalg), AutoEnzyme{Nothing, Nothing}, DifferentiationInterface.Rewrap{0, Tuple{}}}}}, ::Type{BatchDuplicated{Matrix{Float64}, 2}}, tt::Type{Tuple{BatchDuplicated{Matrix{Float64}, 2}}}, ::Val{Enzyme.API.DEM_ForwardMode}, ::Val{2}, ::Val{(false, false)}, ::Val{false}, ::Val{false}, ::Type{FFIABI}, ::Val{true}, ::Val{false})
      @ Enzyme.Compiler ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:9082
   [18] #s2067#19118
      @ ~/.julia/packages/Enzyme/Vjlrr/src/compiler.jl:9219 [inlined]
   [19] var"#s2067#19118"(FA::Any, A::Any, TT::Any, Mode::Any, ModifiedBetween::Any, width::Any, ReturnPrimal::Any, ShadowInit::Any, World::Any, ABI::Any, ErrIfFuncWritten::Any, RuntimeActivity::Any, ::Any, ::Any, ::Any, ::Any, tt::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any)
      @ Enzyme.Compiler ./none:0
   [20] (::Core.GeneratedFunctionStub)(::UInt64, ::LineNumberNode, ::Any, ::Vararg{Any})
      @ Core ./boot.jl:707
   [21] autodiff
      @ ~/.julia/packages/Enzyme/Vjlrr/src/Enzyme.jl:633 [inlined]
   [22] autodiff
      @ ~/.julia/packages/Enzyme/Vjlrr/src/Enzyme.jl:537 [inlined]
   [23] autodiff
      @ ~/.julia/packages/Enzyme/Vjlrr/src/Enzyme.jl:504 [inlined]
   [24] pushforward(::DifferentiationInterface.var"#inner_gradient#46"{typeof(DifferentiationInterfaceTest.arr_to_num_linalg), AutoEnzyme{Nothing, Nothing}, DifferentiationInterface.Rewrap{0, Tuple{}}}, ::DifferentiationInterface.NoPushforwardPrep, ::AutoEnzyme{Nothing, Nothing}, ::Matrix{Float64}, ::Tuple{Matrix{Float64}, Matrix{Float64}})
      @ DifferentiationInterfaceEnzymeExt ~/.julia/packages/DifferentiationInterface/jkKZc/ext/DifferentiationInterfaceEnzymeExt/forward_onearg.jl:75
   [25] pushforward!(::DifferentiationInterface.var"#inner_gradient#46"{typeof(DifferentiationInterfaceTest.arr_to_num_linalg), AutoEnzyme{Nothing, Nothing}, DifferentiationInterface.Rewrap{0, Tuple{}}}, ::Tuple{Matrix{Float64}, Matrix{Float64}}, ::DifferentiationInterface.NoPushforwardPrep, ::AutoEnzyme{Nothing, Nothing}, ::Matrix{Float64}, ::Tuple{Matrix{Float64}, Matrix{Float64}})
      @ DifferentiationInterfaceEnzymeExt ~/.julia/packages/DifferentiationInterface/jkKZc/ext/DifferentiationInterfaceEnzymeExt/forward_onearg.jl:106
   [26] hvp!
      @ ~/.julia/packages/DifferentiationInterface/jkKZc/src/second_order/hvp.jl:244 [inlined]
   [27] hvp!(::typeof(DifferentiationInterfaceTest.arr_to_num_linalg), ::Tuple{Matrix{Float64}, Matrix{Float64}}, ::AutoEnzyme{Nothing, Nothing}, ::Matrix{Float64}, ::Tuple{Matrix{Float64}, Matrix{Float64}})
      @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/jkKZc/src/fallbacks/no_prep.jl:139
   [28] test_correctness(ba::AutoEnzyme{Nothing, Nothing}, scen::Scenario{:hvp, :in, :out, typeof(DifferentiationInterfaceTest.arr_to_num_linalg), Matrix{Float64}, Float64, Tuple{Matrix{Float64}, Matrix{Float64}}, Tuple{}, Matrix{Float64}, Tuple{Matrix{Float64}, Matrix{Float64}}}; isapprox::typeof(isapprox), atol::Int64, rtol::Float64, scenario_intact::Bool)
      @ DifferentiationInterfaceTest ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/tests/correctness_eval.jl:688
   [29] macro expansion
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:115 [inlined]
   [30] macro expansion
      @ ~/.julia/juliaup/julia-1.11.0+0.x64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1700 [inlined]
   [31] macro expansion
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:115 [inlined]
   [32] macro expansion
      @ ~/.julia/juliaup/julia-1.11.0+0.x64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1789 [inlined]
   [33] macro expansion
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:92 [inlined]
   [34] macro expansion
      @ ~/.julia/juliaup/julia-1.11.0+0.x64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1789 [inlined]
   [35] macro expansion
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:90 [inlined]
   [36] macro expansion
      @ ~/.julia/juliaup/julia-1.11.0+0.x64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1789 [inlined]
   [37] macro expansion
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:87 [inlined]
   [38] macro expansion
      @ ~/.julia/juliaup/julia-1.11.0+0.x64.apple.darwin14/share/julia/stdlib/v1.11/Test/src/Test.jl:1700 [inlined]
   [39] test_differentiation(backends::Vector{AutoEnzyme{Nothing, Nothing}}, scenarios::Vector{Scenario}; correctness::Bool, type_stability::Bool, call_count::Bool, sparsity::Bool, detailed::Bool, input_type::Type, output_type::Type, first_order::Bool, second_order::Bool, excluded::Vector{Symbol}, logging::Bool, isapprox::typeof(isapprox), atol::Int64, rtol::Float64, scenario_intact::Bool)
      @ DifferentiationInterfaceTest ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:87
   [40] test_differentiation (repeats 2 times)
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:49 [inlined]
   [41] #test_differentiation#392
      @ ~/.julia/packages/DifferentiationInterfaceTest/nWEdB/src/test_differentiation.jl:144 [inlined]
   [42] top-level scope
      @ ~/Work/GitHub/Julia/DifferentiationInterface.jl/DifferentiationInterface/test/playground.jl:23
   [43] include(mod::Module, _path::String)
      @ Base ./Base.jl:557
   [44] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:323
   [45] _start()
      @ Base ./client.jl:531
[4853] signal 11 (1): Segmentation fault: 11

Relevant test function:

https://github.com/gdalle/DifferentiationInterface.jl/blob/f6632258d24631f60c7aaeb081aea0cf5849b993/DifferentiationInterfaceTest/src/scenarios/default.jl#L173

Related:

wsmoses commented 1 month ago

Don't think so, I think this is just a dup of https://github.com/EnzymeAD/Enzyme.jl/issues/1358