PyJulia crashes when I try to callJulia from inside a Python process using Dask multiprocessing (dask.distributed.Client) throwing the following exceptions:
Calling Julia from Python's built-in multiprocessing works however fine.
I am using Windows 10 (64 bit), Python 3.7, Dask 2.9 and Julia 1.3.
Test Code:
from julia import Julia
import dask.distributed
import concurrent.futures
def jl_calc(id):
func_text = """
function myfunc(n:: Int64)
z:: Int64=1
for i=1:n
z += 2i^2
end
z
end
"""
jl = Julia()
test_func = jl.eval(func_text)
res = test_func(id+1)
return res
def submit_tasks(executor):
futures = []
for i in range(10):
futures.append(executor.submit(jl_calc, i))
results = [future.result() for future in futures]
print(results)
if __name__ == '__main__':
print('calling Julia from Pythons bulit-in Multiprocessing')
executor_multiprocessing = concurrent.futures.ProcessPoolExecutor()
try:
submit_tasks(executor_multiprocessing)
finally:
executor_multiprocessing.shutdown()
print('calling Julia from Pythons bulit-in Multiprocessing successful')
print('calling Julia from Dask Client')
executor_dask = dask.distributed.Client()
try:
submit_tasks(executor_dask)
finally:
executor_dask.shutdown()
print('calling Julia from Dask Client successful')
Hello,
PyJulia crashes when I try to callJulia from inside a Python process using Dask multiprocessing (dask.distributed.Client) throwing the following exceptions:
Calling Julia from Python's built-in multiprocessing works however fine.
I am using Windows 10 (64 bit), Python 3.7, Dask 2.9 and Julia 1.3.
Test Code:
System setup and stack trace is given in the attachment. Dask_Julia_error.txt
Edit:
The same issue has been raised at Dask, because I do not know in which component the issue is:
https://github.com/dask/dask/issues/5732
Best Regards Benjamin