eval in __init__ breaks downstream precompilation

[kolia]

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.3 (2020-11-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(PyMNE_doodle) pkg> st

Project PyMNE_doodle v0.1.0
Status `~/src/tmp/PyMNE_doodle/Project.toml`
  [6c5003b2] PyMNE v0.1.0

shell> cat src/PyMNE_doodle.jl

module PyMNE_doodle
using PyMNE
end

julia> using PyMNE

julia> using PyMNE_doodle

[ Info: Precompiling PyMNE_doodle [c51fb540-dcbe-4563-b378-fed5aac661c3]
ERROR: LoadError: InitError: Evaluation into the closed module `PyMNE` breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating `PyMNE` with `eval` during precompilation - don't do this.
Stacktrace:
 [1] eval at ./boot.jl:331 [inlined]
 [2] __init__() at /home/ubuntu/.julia/packages/PyMNE/pn9jE/src/PyMNE.jl:35
 [3] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:697
 [4] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:782
 [5] _require(::Base.PkgId) at ./loading.jl:1007
 [6] require(::Base.PkgId) at ./loading.jl:928
 [7] require(::Module, ::Symbol) at ./loading.jl:923
 [8] include(::Function, ::Module, ::String) at ./Base.jl:380
 [9] include(::Module, ::String) at ./Base.jl:368
 [10] top-level scope at none:2
 [11] eval at ./boot.jl:331 [inlined]
 [12] eval(::Expr) at ./client.jl:467
 [13] top-level scope at ./none:3
during initialization of module PyMNE
in expression starting at /home/ubuntu/src/tmp/PyMNE_doodle/src/PyMNE_doodle.jl:3
ERROR: Failed to precompile PyMNE_doodle [c51fb540-dcbe-4563-b378-fed5aac661c3] to /home/ubuntu/.julia/compiled/v1.5/PyMNE_doodle/zcqTA_V5NaD.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923

So, I have a PyMNE_doodle project that just depends on PyMNE, that contains a module PyMNE_doodle that just imports PyMNE. I get that lovely error on precompilation of PyMNE_doodle, even though using PyMNE works fine.

[kolia]

https://github.com/beacon-biosignals/PyMNE.jl/blob/bc1a70d720af2a05382c02b1ce857f72354a752c/src/PyMNE.jl#L35 is likely the issue -- thanks @ericphanson

For kicks I changed the module def to

module PyMNE_doodle

function __init__()
    @eval using PyMNE
end

end

and then precompilation works.

[palday]

Interesting. I'm not sure if there is an easy way to avoid this. But at the very least, we can add this information to the documentation.

[ararslan]

This is due to how we fetch the names from the top-level MNE Python namespace to define them at the top level in the Julia module. The problem is that, until the module is initialized (i.e. __init__() is called), the MNE pointer is null, so we don't know what the names are. Thus we have to defer that step until initialization. It's certainly not ideal; definitely open to suggestions for how to do that differently, but in the meantime I'm not sure we can do better. As Phillip said, we can at least document it.