search

JuliaPy / PythonCall.jl

Python and Julia in harmony.
https://juliapy.github.io/PythonCall.jl/stable/
MIT License
722 stars 61 forks source link

Using PythonCall with Distributed #351

Closed martinmestre closed 10 months ago

martinmestre commented 10 months ago

Hello, I would like to know which modification should I do to the following piece of code in order to make PythonCall compatible with Distributed.jl. The problem arises when I have more than one process. 

 @everywhere begin
    pyimport("sys")."path".append("")
    # pushfirst!(pyconvert(Vector{String}, pyimport("sys")."path"), "")
    importLib = pyimport("importlib")
    stream = pyimport("stream")
    potentials = pyimport("potential_classes")
    u = pyimport("astropy.units")
    # importLib.reload(stream)
    # importLib.reload(potentials)
end

The output is: ERROR: Python: Julia: an error occurred while setting an error I guess I am making something wrong with the first line in the begin block. The second (commmented) line works with PyCall. If I avoid both lines, then the program doen't recognize the Python packages "stream". Thank you very much. Best

cjdoris commented 10 months ago

Can you post a full MWE please? i.e. a complete self contained piece of code with all imports etc.

Does the error occur on that first line? i.e. does it still error if you comment out the rest?

How are you launching Julia?

martinmestre commented 10 months ago

Hello, thank for your answer. I found out that there was an error in the astropy version I was using in the cluster. That is why I thought that it was a problem with distributed or slurm. The line pyimport("sys")."path".append("") works well.