@parallel_indices exception in package precompilation

[DanW97]

Hey there!

Having a great time using this package, but I have started to include ParallelStencils into a package that I am writing, and that involves tagging some functions with @parallel_indices. During precompilation to test an example I have written it errors out as I obviously haven't created a stencil yet. What's curious is that functions tagged with @parallel don't cause the same error.

Is there anything I need to declare/ have missed within my own package to stop this error?

Apologies if this is a daft question, I haven't got a lot of experience with GPU stuff!

Cheers,

Dan

[omlins]

Having a great time using this package

Hi @DanW97 , I am glad to hear that!

that involves tagging some functions with @parallel_indices. During precompilation to test an example I have written it errors out as I obviously haven't created a stencil yet.

Could you share the error message, please?

[DanW97]

Yeah sure

ERROR: LoadError: NotInitializedError: no macro or function of the module can be called before @init_parallel_stencil.
Stacktrace:
  [1] check_initialized()
    @ ParallelStencil ~/.julia/packages/ParallelStencil/3flwf/src/init_parallel_stencil.jl:68
  [2] var"@parallel_indices"(__source__::LineNumberNode, __module__::Module, args::Vararg{Any})
    @ ParallelStencil ~/.julia/packages/ParallelStencil/3flwf/src/init_parallel_stencil.jl:5
  [3] #macroexpand#51
    @ ./expr.jl:115 [inlined]
  [4] macroexpand
    @ ./expr.jl:114 [inlined]
  [5] docm(source::LineNumberNode, mod::Module, meta::Any, ex::Any, define::Bool) (repeats 2 times)
    @ Base.Docs ./docs/Docs.jl:537
  [6] (::DocStringExtensions.var"#32#33"{typeof(DocStringExtensions.template_hook)})(::LineNumberNode, ::Vararg{Any})
    @ DocStringExtensions ~/.julia/packages/DocStringExtensions/iscC8/src/templates.jl:11
  [7] var"@doc"(::LineNumberNode, ::Module, ::String, ::Vararg{Any})
    @ Core ./boot.jl:517
  [8] eval
    @ ./boot.jl:373 [inlined]
  [9] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:1196
 [10] include_string(m::Module, txt::String, fname::String)
    @ Base ./loading.jl:1206
 [11] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./essentials.jl:716
 [12] invokelatest(::Any, ::Any, ::Vararg{Any})
    @ Base ./essentials.jl:714
 [13] inlineeval(m::Module, code::String, code_line::Int64, code_column::Int64, file::String; softscope::Bool)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:211
 [14] (::VSCodeServer.var"#60#64"{Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:155
 [15] withpath(f::VSCodeServer.var"#60#64"{Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams}, path::String)
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/repl.jl:184
 [16] (::VSCodeServer.var"#59#63"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:153
 [17] hideprompt(f::VSCodeServer.var"#59#63"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/repl.jl:36
 [18] (::VSCodeServer.var"#58#62"{Bool, Bool, Bool, Module, String, Int64, Int64, String, VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:124
 [19] with_logstate(f::Function, logstate::Any)
    @ Base.CoreLogging ./logging.jl:511
 [20] with_logger
    @ ./logging.jl:623 [inlined]
 [21] (::VSCodeServer.var"#57#61"{VSCodeServer.ReplRunCodeRequestParams})()
    @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:201
 [22] #invokelatest#2
    @ ./essentials.jl:716 [inlined]
 [23] invokelatest(::Any)
    @ Base ./essentials.jl:714
 [24] macro expansion
    @ ~/.vscode/extensions/julialang.language-julia-1.6.10/scripts/packages/VSCodeServer/src/eval.jl:34 [inlined]
 [25] (::VSCodeServer.var"#55#56")()
    @ VSCodeServer ./task.jl:423
in expression starting at /home/dan/Njord/src/collisions.jl:32

The offending function within collisions.jl is

@parallel_indices (ix, iy, iz) function collide_bgk!(ω::T, F::AbstractArray{T}, Feq::AbstractArray{T}) where {T <: AbstractFloat} 
    F[ix, iy, iz, :] -= ω * (F[ix, iy, iz, :] - Feq[ix, iy, iz, :])
    return
end

When running, I ran a script from the examples directory of my package, which has the necessary init commands as are at the top of the files within miniapps and it'd fail during precompilation.

[omlins]

NotInitializedError: no macro or function of the module can be called before @init_parallel_stencil.

This error means that the call to macro @parallel_indices was found before any call to macro @init_parallel_stencil. Note that Julia macros are evaluated simply from top to bottom of a file, independent of the control flow of your application ( just like C pre-processor macros).

It cannot have anything to do with using parallel_indices instead of @parallel, except if there would be a well and long hidden bug.

Should this not allow you to solve your issue, please share a minimal "working" example that allows to reproduce it.