search

FluxML / Zygote.jl

21st century AD
https://fluxml.ai/Zygote.jl/
Other
1.48k stars 210 forks source link

MethodError: no method matching iterate(::Core.CodeInfo) #1449

Open pat-alt opened 1 year ago

pat-alt commented 1 year ago

I'm trying to use Zygote on Julia v1.6.3, because unfortunately that's all that's currently available on the HPC I'm working on. It fails to precompile throwing the following error:

julia> using Zygote
@Info (Base): Precompiling Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]                        
  │                                                                                             
  ╰──────────────────────────────────────────────── 
                       Fri, 18 Aug 2023 08:51:10 
 ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Core.CodeInfo)
Closest candidates are:
  iterate(::Union{LinRange, StepRangeLen}) at range.jl:664
  iterate(::Union{LinRange, StepRangeLen}, ::Int64) at range.jl:664
  iterate(::T) where T<:Union{Base.KeySet{var"#s77", var"#s76"} where {var"#s77", var"#s76"<:Dict}, Base.ValueIterator{var"#s75"} where var"#s75"<:Dict} at dict.jl:693
  ...
Stacktrace:
  [1] indexed_iterate(I::Core.CodeInfo, i::Int64)
    @ Base ./tuple.jl:89
  [2] pullback(::Function, ::Zygote.Context{false}, ::Int64, ::Vararg{Int64, N} where N)
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:44
  [3] pullback(::Function, ::Int64, ::Int64)
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:42
  [4] gradient(::Function, ::Int64, ::Vararg{Int64, N} where N)
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/compiler/interface.jl:96
  [5] top-level scope
    @ ~/.julia/packages/Zygote/4rucm/src/precompile.jl:17
  [6] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [7] include
    @ ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:1 [inlined]
  [8] macro expansion
    @ ~/.julia/packages/Requires/Z8rfN/src/Requires.jl:40 [inlined]
  [9] precompile()
    @ Zygote ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:63
 [10] macro expansion
    @ ~/.julia/packages/PrecompileTools/0yi7r/src/workloads.jl:81 [inlined]
 [11] top-level scope
    @ ~/.julia/packages/Zygote/4rucm/src/Zygote.jl:84
 [12] include
    @ ./Base.jl:386 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1235
 [14] top-level scope
    @ none:1
 [15] eval
    @ ./boot.jl:360 [inlined]
 [16] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [17] top-level scope
    @ none:1
in expression starting at /Users/FA31DU/.julia/packages/Zygote/4rucm/src/precompile.jl:17
in expression starting at /Users/FA31DU/.julia/packages/Zygote/4rucm/src/Zygote.jl:1
ERROR: 
──────────────────────────────────────── ErrorException ────────────────────────────────────────
╭──── Error Stack ─────────────────────────────────────────────────────────────────────────────╮
│                                                                                              │
│  ──────────────────────────────────── In module Base ────────────────────────────────────    │
│                                                                                              │
│         ╭───────────────────────────────────────────────────────────────────────────╮        │
│     (1) │  require(into::Module, mod::Symbol)                                       │        │
│         │  ./loading.jl:923                                                         │        │
│         ╰───────────────────────────────────────────────────────────── TOP LEVEL ───╯        │
│                                                                                              │
│  ────────────────────────────────────────────────────────────────────────────────────────    │
│                                 Skipped 4 frames in Base                                     │
│  ────────────────────────────────────────────────────────────────────────────────────────    │
│                                                                                              │
│         ╭───────────────────────────────────────────────────────────────────────────╮        │
│     (6) │  error(s::String)                                                         │        │
│         │  ./error.jl:33                                                            │        │
│         ╰──────────────────────────────────────────────────────────── ERROR LINE ───╯        │
│                                                                                              │
╰──── Error Stack ─────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────── ErrorException ───────────────────────────────────────╮
│                                                                                              │
│  Failed to precompile Zygote [e88e6eb3-aa80-5325-afca-941959d7151f] to /Users/FA31DU/.jul    │
│  ia/compiled/v1.6/Zygote/jl_chkkNu.                                                          │
│                                                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────╯

Is there anything obvious going on here? Is this salvageable or should I just wait for a newer Julia version to be operational on the HPC?

Thanks!

ToucheSir commented 1 year ago

Nothing obvious, I can't think if any scenario where we'd get back a chunk of IR instead of an actual function here. Can you paste the output of Zygote._pullback(Zygote.Context(), pow, 2, 3), where pow can be copied from https://github.com/FluxML/Zygote.jl/blob/master/src/precompile.jl?

pat-alt commented 1 year ago

I don't think I can properly test this when Zygote fails to precompile? Sorry if I'm missing something 

julia> function pow(x, n)
         r = 1
         while n > 0
           n -= 1
           r *= x
         end
         return r
       end
pow (generic function with 1 method)

julia> Zygote._pullback(Zygote.Context(), pow, 2, 3)
ERROR: 
─────────────────────────────────────────────────────────────── UndefVarError ──────────────────────────────────────────────────────────────
╭──── Error Stack ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                                          │
│      ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮           │
│  (1) │  top-level scope                                                                                                      │           │
│      │  REPL[6]:1                                                                                                            │           │
│      ╰───────────────────────────────────────────────────────────────────────────────────────────────────────── TOP LEVEL ───╯           │
│                                                                                                                                          │
╰──── Error Stack ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────── UndefVarError ─────────────────────────────────────────────────────────────╮
│                                                                                                                                          │
│  UndefVarError: Zygote not defined                                                                                                       │
│                                                                                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

julia>
pat-alt commented 1 year ago

I've just moved that into the precompile.jl. Here's the output (where the stack trace is the same as in the OP):

julia> using Zygote
@Info (Base): Precompiling Zygote [e88e6eb3-aa80-5325-afca-941959d7151f]                                                                                      
  │                                                                                                                                                           
  ╰──────────────────────────────────────────────── 
                       Mon, 21 Aug 2023 14:33:39 
 [ Info: Testing pullback method:
CodeInfo(
1 ─       $(Expr(:meta, :inline))
│   %2  = Base.getfield(args, 1)
│   %3  = Base.getfield(args, 2)
│   %4  = (Vector{Any})()
│   %5  = (Vector{Any})()
│   %6  = (Vector{Any})()
│   %7  = (Vector{Any})()
│         phi_2_1 = %3
│         phi_2_2 = 1
│         phi_2_3 = 0x01
└──       goto #2
    @ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:3 within `pow'
2 ┄ %12 = Zygote._pullback(ctx, Zygote.:>, phi_2_1, 0)
│   %13 = Base.getindex(%12, 1)
│   %14 = Base.getindex(%12, 2)
│         Zygote._push!(%7, phi_2_3)
│         Zygote._push!(%6, %14)
└──       goto #5 if not %13
3 ─       goto #4
    @ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:4 within `pow'
4 ─ %19 = Zygote._pullback(ctx, Zygote.:-, phi_2_1, 1)
│   %20 = Base.getindex(%19, 1)
│   %21 = Base.getindex(%19, 2)
│   @ /Users/patrickaltmeyer/.julia/packages/Zygote/4rucm/src/precompile.jl:5 within `pow'
│   %22 = Zygote._pullback(ctx, Zygote.:*, phi_2_2, %2)
│   %23 = Base.getindex(%22, 1)
│   %24 = Base.getindex(%22, 2)
│         Zygote._push!(%5, %24)
│         Zygote._push!(%4, %21)
│         phi_2_1 = %20
│         phi_2_2 = %23
│         phi_2_3 = 0x02
└──       goto #2
5 ─ %31 = Base.tuple(%7, %6, %5, %4)
│   %32 = (Zygote.Pullback{Tuple{typeof(Zygote.pow), Int64, Int64}, Any})(%31)
│   %33 = Base.tuple(phi_2_2, %32)
└──       return %33
)
ERROR: LoadError: LoadError: MethodError: no method matching iterate(::Core.CodeInfo)
Closest candidates are:
  iterate(::Union{LinRange, StepRangeLen}) at range.jl:664
  iterate(::Union{LinRange, StepRangeLen}, ::Int64) at range.jl:664
  iterate(::T) where T<:Union{Base.KeySet{var"#s77", var"#s76"} where {var"#s77", var"#s76"<:Dict}, Base.ValueIterator{var"#s75"} where var"#s75"<:Dict} at dict.jl:693
  ...
Stacktrace:
[...]
ToucheSir commented 1 year ago

Do you mind testing on a later 1.6 version? It seems to me that our CI should've caught this if LTS is failing, so perhaps some 1.6 patch release fixed internals so that Zygote works again.

pat-alt commented 1 year ago

I think the CI runs tests on v1.6.7, which works just fine for me locally:

julia> versioninfo()
Julia Version 1.6.7
Commit 3b76b25b64 (2022-07-19 15:11 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin21.4.0)
  CPU: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = code

julia> using Zygote

julia>
ToucheSir commented 1 year ago

That's good. I think it'd probably be easier and more robust to see if you can get a newer patch/minor release of Julia than for us to add a workaround for issues which appear to have been fixed in a patch release. I'll leave this open for a bit though in case anyone does discover the root cause and it happens to be a straightforward fix.

pat-alt commented 1 year ago

Sounds good, thank you!