IntelLabs / ParallelAccelerator.jl

The ParallelAccelerator package, part of the High Performance Scripting project at Intel Labs
BSD 2-Clause "Simplified" License
294 stars 32 forks source link

DomainIR error for HPAT kmeans #116

Closed ehsantn closed 7 years ago

ehsantn commented 8 years ago

DomainIR throws an error for kmeans example of HPAT on Julia 0.5-rc3.

ERROR: LoadError: Expected one method from call to Base._methods in lambdaTypeinf to match type Tuple{Base.#-,Float64,Any}, but none: Any[svec(Tuple{Base.#-,Float64,Float64},svec(),-(x::Float64, y::Float64) at float.jl:242),svec(Tuple{Base.#-,Float64,Complex{Bool}},svec(),-(x::Real, z::Complex{Bool}) at complex.jl:154),svec(Tuple{Base.#-,Float64,Complex},svec(),-(x::Real, z::Complex) at complex.jl:166),svec(Tuple{Base.#-,Float64,BigFloat},svec(),-(c::Union{Float16,Float32,Float64}, x::BigFloat) at mpfr.jl:284),svec(Tuple{Base.#-,Float64,Float64},svec(Float64),-{T<:Number}(x::T, y::T) at promotion.jl:257),svec(Tuple{Base.#-,Float64,Number},svec(),-(x::Number, y::Number) at promotion.jl:192),svec(Tuple{Base.#-,Float64,AbstractArray},svec(),-(x::Number, A::AbstractArray) at arraymath.jl:97),svec(Tuple{Base.#-,Float64,UniformScaling},svec(),-(x::Number, J::UniformScaling) at linalg/uniformscaling.jl:49)]
 in #lambdaTypeinf#21(::Bool, ::Function, ::Type{T}, ::Tuple{DataType,DataType}) at /Users/ehsan/.julia/v0.5/CompilerTools/src/lambda.jl:884
 in (::CompilerTools.LambdaHandling.#kw##lambdaTypeinf)(::Array{Any,1}, ::CompilerTools.LambdaHandling.#lambdaTypeinf, ::Type{T}, ::Tuple{DataType,DataType}) at ./<missing>:0
 in get_ast_for_lambda(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Base.#-, ::Array{Type,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1814
 in translate_call_mapop(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1764
 in translate_call_symbol(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1560
 in translate_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1263
 in from_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1247
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2615
 in from_exprs(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1102
 in normalize_args(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1275
 in translate_call_symbol(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1549
 in translate_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1263
 in from_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1247
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2615
 in from_exprs(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1102
 in normalize_args(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1275
 in translate_call_reduceop(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2309
 in translate_call_symbol(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1587
 in translate_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1263
 in from_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1247
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2615
 in from_exprs(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1102
 in normalize_args(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1275
 in translate_call_symbol(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1549
 in translate_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1263
 in from_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1247
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2615
 in from_assignment(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1205
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2611
 in from_body(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1121
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2609
 in from_lambda(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::LambdaInfo, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1079
 in get_ast_for_lambda(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1818
 in get_lambda_for_arg(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::SSAValue, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1916
 in translate_call_cartesianarray(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2244
 in translate_call_symbol(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1573
 in translate_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Type{T}, ::Symbol, ::Any, ::Array{Any,1}, ::Symbol, ::Array{Any,1}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1263
 in from_call(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1247
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2615
 in from_assignment(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1205
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2611
 in from_body(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1121
 in from_expr(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::Expr) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2609
 in from_lambda(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::LambdaInfo, ::Void) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:1079
 in from_root(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::LambdaInfo) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2536
 in collect(::Base.Generator{Array{Any,1},ParallelAccelerator.DomainIR.##103#104{ParallelAccelerator.DomainIR.IRState,ParallelAccelerator.DomainIR.IREnv}}) at ./array.jl:304
 in from_expr_tiebreak(::ParallelAccelerator.DomainIR.IRState, ::ParallelAccelerator.DomainIR.IREnv, ::LambdaInfo) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2541
 in from_expr(::Module, ::LambdaInfo) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/domain-ir.jl:2550
 in toDomainIR(::GlobalRef, ::LambdaInfo, ::Tuple{DataType,DataType,DataType}) at /Users/ehsan/.julia/v0.5/ParallelAccelerator/src/driver.jl:177
 in processFuncCall(::Any, ::Any, ::Any) at /Users/ehsan/.julia/v0.5/CompilerTools/src/OptFramework.jl:442
 in kmeans(::Int64, ::Int64, ::String) at /Users/ehsan/.julia/v0.5/CompilerTools/src/OptFramework.jl:557
 in main() at /Users/ehsan/.julia/v0.5/HPAT/examples/kmeans.jl:90
 in include_from_node1(::String) at ./loading.jl:426
 in include_from_node1(::String) at /Users/ehsan/Downloads/julia-0.5.0-rc3/usr/lib/julia/sys.dylib:?
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
 in _start() at /Users/ehsan/Downloads/julia-0.5.0-rc3/usr/lib/julia/sys.dylib:?
while loading /Users/ehsan/.julia/v0.5/HPAT/examples/kmeans.jl, in expression starting on line 104
ninegua commented 8 years ago

looks like the said function was called with Any type for its second argument. Please double check if the source had any thing with Any type.

ehsantn commented 8 years ago

The program hasn't changed from before. input parameters are typed properly also.

Looks like the second input of the subtraction in the first comprehension is not typed properly. Could it because of : operator in ParallelAccelerator.API.getindex?

SSAValue(7) = (ParallelAccelerator.API.sqrt)((ParallelAccelerator.API.sum)(($(Expr(:invoke, LambdaInfo for getindex(::Array{Float64,2}, ::Colon, ::Vararg{Any,N}), :(ParallelAccelerator.API.getindex), :((Core.getfield)(_1,:points)::Array{Float64,2}), :(Main.:), :(_2))) - (ParallelAccelerator.API.getindex)((Core.getfield)((Core.getfield)(_1,:centroids)::Core.Box,:contents),Main.:,_7)) .^ 2))
@acc hpat function kmeans(numCenter, iterNum, file_name)
    points = DataSource(Matrix{Float64},HDF5,"/points", file_name)
    D,N = size(points) # number of features, instances
    centroids = rand(D, numCenter)

    for l in 1:iterNum
        dist = [ Float64[sqrt(sum((points[:,i]-centroids[:,j]).^2)) for j in 1:numCenter] for i in 1:N]
        labels = Int[indmin(dist[i]) for i in 1:N]
        centroids = Float64[ sum(points[j,labels.==i])/sum(labels.==i) for j in 1:D, i in 1:numCenter]
    end 
    return centroids
end
ninegua commented 7 years ago

Once we bring the k-means.jl source from HAT to match the one in ParallelAccelerator, the inference issue is gone.

Our comprehension translation relies on some type hack for Julia 0.5, so it is advised to annotate the LHS with precise type in case something fails due to Any type.

After the change, now it triggers some other error further down in HPAT pipeline. You may want to take a look. @ehsantn

ehsantn commented 7 years ago

It's sad that we need this much type annotation now. I will investigate the HPAT problems.

ehsantn commented 7 years ago

I think this problem is resolved.