timholy / MethodAnalysis.jl

Utilities to analyze Julia's method tables
https://timholy.github.io/MethodAnalysis.jl/stable/
MIT License
94 stars 3 forks source link

TypeError: in Type, in parameter, expected Type, got Vararg #44

Closed thchr closed 1 year ago

thchr commented 1 year ago

I'm not sure if this is me or the package, but will report anyway:

julia> using MethodAnalysis
julia> mis = methodinstances();
julia> argmatch(typs) = length(typs) >= 2 && typs[2] === AbstractArray;
julia> findcallers(convert, argmatch, mis)
ERROR: TypeError: in Type, in parameter, expected Type, got Vararg
Stacktrace:
 [1] findcallers(f::Function, argmatch::typeof(argmatch), mis::Vector{Core.MethodInstance}; callhead::Symbol, world::UInt64, interp::Core.Compiler.NativeInterpreter)
   @ MethodAnalysis ~/.julia/packages/MethodAnalysis/h6wmZ/src/findcallers.jl:168
 [2] findcallers(f::Function, argmatch::typeof(argmatch), mis::Vector{Core.MethodInstance})
   @ MethodAnalysis ~/.julia/packages/MethodAnalysis/h6wmZ/src/findcallers.jl:98
 [3] top-level scope
   @ REPL[62]:1

Which points to /src/findcallers.jl:168. The same thing works fine if I swap out convert for e.g. show.

Tested on Julia v1.8 and v1.9-rc2.

timholy commented 1 year ago

I can't reproduce this on either 1.8.5 or 1.9-rc2. Do you have additional packages in your environment?

thchr commented 1 year ago

Not to my knowledge, e.g., I see this also when activating a temporary environment:

]               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.5 (2023-01-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.8) pkg> activate --temp
  Activating new project at `/tmp/jl_NXO4xB`

(jl_NXO4xB) pkg> add MethodAnalysis
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `/tmp/jl_NXO4xB/Project.toml`
  [85b6ec6f] + MethodAnalysis v0.4.12
    Updating `/tmp/jl_NXO4xB/Manifest.toml`
  [1520ce14] + AbstractTrees v0.4.4
  [85b6ec6f] + MethodAnalysis v0.4.12

julia> using MethodAnalysis

julia> mis = methodinstances();

julia> findcallers(convert, argmatch2, mis)^C

julia> argmatch(typs) = length(typs) >= 2 && typs[2] === AbstractArray;

julia> findcallers(convert, argmatch, mis)
ERROR: TypeError: in Type, in parameter, expected Type, got Vararg
Stacktrace:
 [1] findcallers(f::Function, argmatch::typeof(argmatch), mis::Vector{Core.MethodInstance}; callhead::Symbol, world::UInt64, interp::Core.Compiler.NativeInterpreter)
   @ MethodAnalysis ~/.julia/packages/MethodAnalysis/h6wmZ/src/findcallers.jl:168
 [2] findcallers(f::Function, argmatch::typeof(argmatch), mis::Vector{Core.MethodInstance})
   @ MethodAnalysis ~/.julia/packages/MethodAnalysis/h6wmZ/src/findcallers.jl:98
 [3] top-level scope
   @ REPL[6]:1

(jl_NXO4xB) pkg> st
Status `/tmp/jl_NXO4xB/Project.toml`
  [85b6ec6f] MethodAnalysis v0.4.12
timholy commented 1 year ago

I might be able to come up with a fix, but I don't know how to write a test; for that it would be nice to catch the source of the error. Since I can't debug this, can you? Here's a potential head-start:

diff --git a/src/findcallers.jl b/src/findcallers.jl
index 18d034c..d7872ea 100644
--- a/src/findcallers.jl
+++ b/src/findcallers.jl
@@ -165,7 +165,11 @@ function findcallers(f, argmatch::Union{Function,Nothing}, mis::AbstractVector{C
                             push!(argtypes, Core.Typeof(getfield(a.mod, a.name)))
                         elseif isexpr(a, :static_parameter)
                             a = a::Expr
-                            push!(argtypes, Type{sparams[a.args[1]::Int]})
+                            T = sparams[a.args[1]::Int]
+                            if Base.isvarargtype(T)
+                                @show T src i item
+                            end
+                            push!(argtypes, Type{T})
                         else
                             push!(argtypes, extract(a, sparams))
                         end
thchr commented 1 year ago

Definitely. @eval'ing the diff above into MethodAnalysis, I get the following output (now on v1.9-rc2 and in my default environment):

julia> findcallers(convert, argmatch, mis)
T = Vararg
src = CodeInfo(
     @ /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/Pkg/src/Versions.jl:14 within `VersionBound`
1 ── %1  = ($(Expr(:static_parameter, 1)) <= 3)::Bool
└───       goto #3 if not %1
2 ──       goto #4
3 ── %4  = Pkg.Versions.ArgumentError("VersionBound: you can only specify major, minor and patch versions")::Core.Const(ArgumentError("VersionBound: you can only specify major, minor and patch versions"))
└───       Pkg.Versions.throw(%4)::Union{}
     @ /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/Pkg/src/Versions.jl:15 within `VersionBound`
4 ┄─ %6  = ($(Expr(:static_parameter, 1)) == 0)::Bool
└───       goto #6 if not %6
5 ── %8  = Pkg.Versions.VersionBound::Core.Const(Pkg.Versions.VersionBound)
│    %9  = Core.fieldtype(%8, 1)::Core.Const(Tuple{UInt32, UInt32, UInt32})
│    %10 = Core.tuple(0, 0, 0)::Core.Const((0, 0, 0))
│    %11 = Base.convert(%9, %10)::Core.Const((0x00000000, 0x00000000, 0x00000000))
│    %12 = Core.fieldtype(%8, 2)::Core.Const(Int64)
│    %13 = Base.convert(%12, $(Expr(:static_parameter, 1)))::Int64
│    %14 = %new(%8, %11, %13)::Core.PartialStruct(Pkg.Versions.VersionBound, Any[Core.Const((0x00000000, 0x00000000, 0x00000000)), Int64])
└───       return %14
     @ /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/Pkg/src/Versions.jl:16 within `VersionBound`
6 ── %16 = ($(Expr(:static_parameter, 1)) == 1)::Bool
└───       goto #8 if not %16
7 ── %18 = Pkg.Versions.VersionBound::Core.Const(Pkg.Versions.VersionBound)
│    %19 = Core.fieldtype(%18, 1)::Core.Const(Tuple{UInt32, UInt32, UInt32})
│    %20 = Base.getindex(tin, 1)::Int64
│    %21 = Core.tuple(%20, 0, 0)::Core.PartialStruct(Tuple{Int64, Int64, Int64}, Any[Int64, Core.Const(0), Core.Const(0)])
│    %22 = Base.convert(%19, %21)::Core.PartialStruct(Tuple{UInt32, UInt32, UInt32}, Any[UInt32, Core.Const(0x00000000), Core.Const(0x00000000)])
│    %23 = Core.fieldtype(%18, 2)::Core.Const(Int64)
│    %24 = Base.convert(%23, $(Expr(:static_parameter, 1)))::Int64
│    %25 = %new(%18, %22, %24)::Core.PartialStruct(Pkg.Versions.VersionBound, Any[Core.PartialStruct(Tuple{UInt32, UInt32, UInt32}, Any[UInt32, Core.Const(0x00000000), Core.Const(0x00000000)]), Int64])
└───       return %25
     @ /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/Pkg/src/Versions.jl:17 within `VersionBound`
8 ── %27 = ($(Expr(:static_parameter, 1)) == 2)::Bool
└───       goto #10 if not %27
9 ── %29 = Pkg.Versions.VersionBound::Core.Const(Pkg.Versions.VersionBound)
│    %30 = Core.fieldtype(%29, 1)::Core.Const(Tuple{UInt32, UInt32, UInt32})
│    %31 = Base.getindex(tin, 1)::Int64
│    %32 = Base.getindex(tin, 2)::Int64
│    %33 = Core.tuple(%31, %32, 0)::Core.PartialStruct(Tuple{Int64, Int64, Int64}, Any[Int64, Int64, Core.Const(0)])
│    %34 = Base.convert(%30, %33)::Core.PartialStruct(Tuple{UInt32, UInt32, UInt32}, Any[UInt32, UInt32, Core.Const(0x00000000)])
│    %35 = Core.fieldtype(%29, 2)::Core.Const(Int64)
│    %36 = Base.convert(%35, $(Expr(:static_parameter, 1)))::Int64
│    %37 = %new(%29, %34, %36)::Core.PartialStruct(Pkg.Versions.VersionBound, Any[Core.PartialStruct(Tuple{UInt32, UInt32, UInt32}, Any[UInt32, UInt32, Core.Const(0x00000000)]), Int64])
└───       return %37
     @ /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/Pkg/src/Versions.jl:18 within `VersionBound`
10 ─ %39 = ($(Expr(:static_parameter, 1)) == 3)::Bool
└───       goto #12 if not %39
11 ─ %41 = Pkg.Versions.VersionBound::Core.Const(Pkg.Versions.VersionBound)
│    %42 = Core.fieldtype(%41, 1)::Core.Const(Tuple{UInt32, UInt32, UInt32})
│    %43 = Base.getindex(tin, 1)::Int64
│    %44 = Base.getindex(tin, 2)::Int64
│    %45 = Base.getindex(tin, 3)::Int64
│    %46 = Core.tuple(%43, %44, %45)::Tuple{Int64, Int64, Int64}
│    %47 = Base.convert(%42, %46)::Tuple{UInt32, UInt32, UInt32}
│    %48 = Core.fieldtype(%41, 2)::Core.Const(Int64)
│    %49 = Base.convert(%48, $(Expr(:static_parameter, 1)))::Int64
│    %50 = %new(%41, %47, %49)::Pkg.Versions.VersionBound
└───       return %50
     @ /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/Pkg/src/Versions.jl:19 within `VersionBound`
12 ─ %52 = Base.string("invalid ", $(Expr(:static_parameter, 1)))::String
│          Pkg.Versions.error(%52)::Union{}
└───       Core.Const(:(return %53))::Union{}
)
i = 3
item = MethodInstance for Pkg.Versions.VersionBound(::Tuple{Int64, Vararg{Int64}})
ERROR: TypeError: in Type, in parameter, expected Type, got Vararg
Stacktrace:
 [1] findcallers(f::Function, argmatch::typeof(argmatch), mis::Vector{Core.MethodInstance}; callhead::Symbol, world::UInt64, interp::Core.Compiler.NativeInterpreter)
   @ MethodAnalysis ./REPL[16]:76
 [2] findcallers(f::Function, argmatch::typeof(argmatch), mis::Vector{Core.MethodInstance})
   @ MethodAnalysis ./REPL[16]:2
 [3] top-level scope
   @ REPL[17]:1
timholy commented 1 year ago

Interesting! On my machine, 1.9-rc2, I get this

julia> m = only(methods(Pkg.Versions.VersionBound, (Tuple,)))
Pkg.Versions.VersionBound(tin::Tuple{Vararg{Integer, n}}) where n
     @ Pkg.Versions ~/.julia/juliaup/julia-1.9.0-rc2+0.x64.linux.gnu/share/julia/stdlib/v1.9/Pkg/src/Versions.jl:13

julia> mis = methodinstances(m)
7-element Vector{Core.MethodInstance}:
 MethodInstance for Pkg.Versions.VersionBound(::Tuple{})
 MethodInstance for Pkg.Versions.VersionBound(::Tuple{Int64})
 MethodInstance for Pkg.Versions.VersionBound(::Tuple{Int64, Int64})
 MethodInstance for Pkg.Versions.VersionBound(::Tuple{Int64, Int64, Int64})
 MethodInstance for Pkg.Versions.VersionBound(::Tuple{UInt32, UInt32, UInt32})
 MethodInstance for Pkg.Versions.VersionBound(::Tuple{UInt32, UInt32})
 MethodInstance for Pkg.Versions.VersionBound(::Tuple{UInt32})

So there is no precompiled specialization for Tuple{Int64, Vararg{Int64}}. However, I can force it:

julia> mi = Core.Compiler.specialize_method(m, Tuple{Type{Pkg.Versions.VersionBound}, Tuple{Int, Vararg{Int}}}, Core.svec())
MethodInstance for Pkg.Versions.VersionBound(::Tuple{Int64, Vararg{Int64}})

julia> findcallers(convert, argmatch, [mi])
ERROR: TypeError: in Type, in parameter, expected Type, got Vararg
...
thchr commented 1 year ago

Could the discrepant specialization of Pkg.Versions.VersionBound on my end have been created "from" my default environment before I switched to the temporary environment?

timholy commented 1 year ago

Given your comment earlier about not having other packages loaded, presumably you disabled any startup. So that seems likely, simply by the process of elimination.