Make it possible to use DaemonMode.jl

[linkret]

I'm having an issue because my PC is old, so initializing a new Julia REPL takes 20-30 seconds. For my Python TKInter GUI this means that the window doesn't even show until Julia is initialized. I've been using PyJulia like this:

compiled_modules_flag = '-c' in sys.argv

if compiled_modules_flag:
    jl = Julia(compiled_modules=True, sysimage="solutionEvaluation.so")
else:   
    jl = Julia(compiled_modules=False)

Main.include("solutionEvaluation.jl")

It works, and I'm able to call the methods from my Julia script, but I wanted to make it faster. Compiling the script (which took me over 7 hours) didn't seem to help at all - in fact, it makes the loading time even worse by 2-3 seconds, and the .so is over 1GB in size.

I found the following post and tried to use DaemonMode: https://dmolina.github.io/DaemonMode.jl/stable/

The idea is to completely skip the Julia loading time, because it's manually preloaded in the background persistently (ran from another console).

However, I'm not able to integrate this with PyJulia, if it's even possible. I took a look at the source code at core.py, options.py, and libjulia.py, and there doesn't seem to be a way to pass an additional list of arguments to the Julia Repl. The "command" that I wished to execute to use DaemonMode is: julia -e 'using DaemonMode; runargs()' program.jl - which doesn't start a new Julia REPL, and instead uses the one from the Daemon running in my background.

Currently I'm only able to get it to work from Python by using subprocess() calls like this:

result = subprocess.run(
    ["julia", "-e", "using DaemonMode; runargs()", "solutionEvaluation.jl"],
    capture_output=True,
    text=True
)

This makes my GUI pop up after only 1 or 2 seconds, since a new Julia REPL doesn't need to be loaded.

However this means I'm not using PyJulia at all, and I have to manually call my Julia methods through other subprocess runs(), and interpret the results from raw strings. It's doable, but really inelegant.

So I was wondering is there a way to do this with PyJulia? If not, is it possible to add it, or too complicated and not worth it? I think anybody with a weak PC like me would greatly benefit from using a Daemon to skip Julia loading times when using PyJulia. Sorry if this question isn't a real Issue, I'm new to this repo and Julia in general. Cheers.

[linkret]

Also for anyone wandering I'm invoking Julia methods like this from python with DaemonMode:

result = subprocess.run(
  ["julia", "-e", "using DaemonMode; runexpr(\"evaluate_and_draw(\\\"solution.txt\\\")\")"],
  capture_output=True,
  text=True
)

It's a little slower than PyJulia's Main.evaluate_and_draw() call was, but overall better for me.

[mkitti]

Have you considered using https://github.com/JuliaPy/PythonCall.jl / juliacall ?

[linkret]

No, I didn't know that repo existed, thanks for the link and the reply!

From what I can tell, and please correct me if I'm wrong, already calling the import statement like this:

from juliacall import Main as jl

, will create a new Julia REPL - which is the step that I wish to skip (or, well, substitute with DaemonMode). So I'm not sure how I could use JuliaCall in my Python script without waiting 20+ seconds for Julia to startup.

[mkitti]

I see what you mean now. What you are suggesting is really something quite distinct from what either of those packages do. That would involve running Julia as a separate process. This package focuses on running Julia and Python in the same process.

What you really want to do here is to do some sort of remote procedure call. I would consider using something like ZMQ via

• https://github.com/zeromq/pyzmq
• https://github.com/JuliaInterop/ZMQ.jl

You may also want to consider using Apache Arrow to exchange information:

• https://arrow.apache.org/docs/python/index.html
• https://arrow.apache.org/julia/stable/

This is far beyond the scope of this package.

[linkret]

Okay, thanks