omlins / ParallelStencil.jl

Package for writing high-level code for parallel high-performance stencil computations that can be deployed on both GPUs and CPUs
BSD 3-Clause "New" or "Revised" License
322 stars 38 forks source link

@parallel_indices exception in package precompilation #50

Closed DanW97 closed 1 year ago

DanW97 commented 2 years ago

Hey there!

Having a great time using this package, but I have started to include ParallelStencils into a package that I am writing, and that involves tagging some functions with @parallel_indices. During precompilation to test an example I have written it errors out as I obviously haven't created a stencil yet. What's curious is that functions tagged with @parallel don't cause the same error.

Is there anything I need to declare/ have missed within my own package to stop this error?

Apologies if this is a daft question, I haven't got a lot of experience with GPU stuff!

Cheers,

Dan

omlins commented 2 years ago

Having a great time using this package

Hi @DanW97 , I am glad to hear that!

that involves tagging some functions with @parallel_indices. During precompilation to test an example I have written it errors out as I obviously haven't created a stencil yet.

Could you share the error message, please?

DanW97 commented 2 years ago

Yeah sure


ERROR: LoadError: NotInitializedError: no macro or function of the module can be called before @init_parallel_stencil.
Stacktrace:
  [1] check_initialized()
    @ ParallelStencil ~/.julia/packages/ParallelStencil/3flwf/src/init_parallel_stencil.jl:68
  [2] var"@parallel_indices"(__source__::LineNumberNode, __module__::Module, args::Vararg{Any})
    @ ParallelStencil ~/.julia/packages/ParallelStencil/3flwf/src/init_parallel_stencil.jl:5
  [3] #macroexpand#51
    @ ./expr.jl:115 [inlined]
  [4] macroexpand
    @ ./expr.jl:114 [inlined]
  [5] docm(source::LineNumberNode, mod::Module, meta::Any, ex::Any, define::Bool) (repeats 2 times)
    @ Base.Docs ./docs/Docs.jl:537
  [6] (::DocStringExtensions.var"#32#33"{typeof(DocStringExtensions.template_hook)})(::LineNumberNode, ::Vararg{Any})
    @ DocStringExtensions ~/.julia/packages/DocStringExtensions/iscC8/src/templates.jl:11
  [7] var"@doc"(::LineNumberNode, ::Module, ::String, ::Vararg{Any})
    @ Core ./boot.jl:517
  [8] eval
    @ ./boot.jl:373 [inlined]
  [9] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:1196
 [10] include_string(m::Module, txt::String, fname::String)
    @ Base ./loading.jl:1206
 [11] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./essentials.jl:716
 [12] invokelatest(::Any, ::Any, ::Vararg{Any})
    @ Base ./essentials.jl:714
 [13] inlineeval(m::Module, code::String, code_line::Int64, code_column::Int64, file::String; softscope::Bool)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:211
 [14] (::VSCodeServer.var"#60#64"{Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:155
 [15] withpath(f::VSCodeServer.var"#60#64"{Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams}, path::String)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/repl.jl:184
 [16] (::VSCodeServer.var"#59#63"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:153
 [17] hideprompt(f::VSCodeServer.var"#59#63"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/repl.jl:36
 [18] (::VSCodeServer.var"#58#62"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:124
 [19] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging ./logging.jl:511
 [20] with_logger
    @ ./logging.jl:623 [inlined]
 [21] (::VSCodeServer.var"#57#61"{VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:201
 [22] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
 [23] invokelatest(::Any)
    @ Base ./essentials.jl:714
 [24] macro expansion
    @ ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:34 [inlined]
 [25] (::VSCodeServer.var"#55#56")()
    @ VSCodeServer ./task.jl:423
in expression starting at /home/dan/Njord/src/collisions.jl:32

The offending function within collisions.jl is

@parallel_indices (ix, iy, iz) function collide_bgk!(ω::T, F::AbstractArray{T}, Feq::AbstractArray{T}) where {T <: AbstractFloat} 
    F[ix, iy, iz, :] -= ω * (F[ix, iy, iz, :] - Feq[ix, iy, iz, :])
    return
end

When running, I ran a script from the examples directory of my package, which has the necessary init commands as are at the top of the files within miniapps and it'd fail during precompilation.

omlins commented 2 years ago

NotInitializedError: no macro or function of the module can be called before @init_parallel_stencil.

This error means that the call to macro @parallel_indices was found before any call to macro @init_parallel_stencil. Note that Julia macros are evaluated simply from top to bottom of a file, independent of the control flow of your application ( just like C pre-processor macros).

It cannot have anything to do with using parallel_indices instead of @parallel, except if there would be a well and long hidden bug.

Should this not allow you to solve your issue, please share a minimal "working" example that allows to reproduce it.