Memory Leak Fix Template

[zenna]

Belong is a template of how to address memory leak problems

# The problem is that Julia does not garbage collect modules
# This means the following code will leak memory

using Random

randsym() = Symbol(randstring('a':'z'))

function test_memory_leak()
  while true
    m = Module(randsym())
    x = rand()
    Base.eval(m, :(f() = rand() + $x))
    @timed Base.invokelatest(m.f)
    # m not garbage collected
  end
end

# There's no way to make modules be garbaged collected without changing Julia
# Which I don't want to do.

# Alternative: eval in  another process then kill process
# Open julia with multiple processes 
# e.g. julia -p 4

using Distributed

function no_leak()
  while true
    # Add new process
    procs = addprocs(1)

    # Make the model
    modname = randsym()
    expr = :(module $modname
        f() = rand()
      end)
    Distributed.remotecall_eval(Main, procs, expr)
    callexpr = :(($modname).f())
    @show Distributed.remotecall_eval(Main, procs..., callexpr)

    # Cleanup -- delete process
    rmprocs(procs...)
  end
end

Creating and deleting a new julia process is very slow. A more sophisticated version of this would run many (but not too many) tasks on a process before killing it, and eliding the cost of creating and destroying

@riadas