More flexible module initialization interface

[yuyichao]

Currently the runtime-only module initialization is done by defining a single __init__ function in each module. This means that the initialization of a module have to be done in a single function (and file).

However, IMHO, it would be better if the initialization code can be distributed across different position. This can make the organization of the initialization code better (and ppl are already inventing their own way to achieve this in packages e.g. in Nettle.jl). This should also help macros (or otherwise generated code) to use this function. For example, the deps.jl generated by BinDeps checks the existance of the library as well as returning the path to the library. One of the function (returning the path) needs to be done at compile time while the other needs to be done at runtime. It is probably possible in this case for BinDeps to define a initialization hook to be called by user's __init__ but it can make things much more complicated and such workaround would be hard to implement for a macro.

IMHO this interface can be similar to atexit or atreplinit (i.e. define a function in each module to register the initialization function) or similar to @inline or @generated (to mark the function as a initializer).

[vtjnash]

BinDeps provides the flexibility to define an initialization hook (https://github.com/JuliaLang/Gtk.jl/blob/8318ad3481acf30cf4ccecc2f984a27f8ce324a5/deps/build.jl#L42-L51)

i think my only question with an atinit-like function is whether it is possible to run these checks in an arbitrary order, and if not, what order should they run

[yuyichao]

AFAICT @checked_lib is not using an initialization hook so that check will always be done at compile time and therefore the preload can only be run a compile time as well. I also didn't find a way to add hooks for system libs.

I think the order to run them should be same with the order they are registered since this is the order they will be run if they were just normal code. If a init hook is registered after the initialization of a module, it should be run immediately so that the author of macros/generated code don't have to deal with this situation..

[cstjean]

+1 to this. There would be a number of use-cases in PyCall.jl. Common Lisp has eval-when, and while it's not pretty, it does the job.

[timholy]

See also https://github.com/timholy/Images.jl/pull/443.

[simonbyrne]

This would be really useful for a couple of cases I'm working on:

In MPI.jl, I'd like to have a mechanism where package develoers can declare custom MPI datatypes in their own modules, and have it auto-initialize when the module is loaded:

module MyPkg
struct MyType
  ...
end

@mpitype MyType
end

where @mpitype MyType expands to something like

let dt = MPI.Datatype()
   MPI.Datatype(::Type{MyType}) = dt
   atinit() do
      MPI.create!(dt, MyType)
   end
end

Currently there's not a way to do this without requiring package developers to insert hooks into their __init__() functions.

In NVTX.jl, I would like to have a macro which when used returns a module-specific domain, and which is initialized when the module is initialized:

NVTX.@domain

would be something like:

macro domain()
   if !isdefined(__module__, :__nvtx_domain__)
      domain = Domain()
      @eval __module__ begin
          const __nvtx_domain__ = $domain
          atinit() do
               NVTX.init!($domain, $(string(__module__)))
          end
      end
   end
   $__module__.__nvtx_domain__
end