johnmchambers / XRJulia

XR-style Interface to Julia (from "Extending R")
33 stars 2 forks source link

Error when creating a new evaluator with specified port #16

Closed oliviaAB closed 6 years ago

oliviaAB commented 6 years ago

Hi,

I am trying to create a new Julia evaluator on a specified port number myportid using RJulia(port = as.integer(myportid), .makeNew = T) or XRJulia::JuliaInterface(port = as.integer(myportid)), but every time this throws the following error:

Error in .Object$initialize(...) : 
  Unable to start Julia connection on port 1319 : all connections are in use

even if I choose a port number equal to 1 + the port number of the previous evaluator (what is done by default) and even though I don't have any connection on this port. Is there any way to specify manually the port number for an evaluator?

The idea would be to run a code in parallel in R using mclapply, and for each iteration create a new Julia evaluator on which I can run my computations. I cannot simply use RJulia(.makeNew = T) otherwise the different R threads try to open a connection on the same port (as the evalutor creation runs in parallel).

Thanks for your help!

johnmchambers commented 6 years ago

Definitely a bug: supplying a port number causes that failure.

Will look into it (may be a little while).

It would seem that a workaround is to include the Julia evaluators as part of the argument X to mclapply; i.e., each element of X is a list whose first element, say, is the evaluator to use for interface calls to julia in the thread that corresponds to that element. Then all the evaluators can be generated in the main by repeatedly using RJulia(.makeNew = TRUE)

johnmchambers commented 6 years ago

PS on my previous comment. Not a reasonable idea, since even if it worked, all the Julia work would be done in the main thread.

oliviaAB commented 6 years ago

Thank you for your answer!

I am not sure I understand your comment:

Not a reasonable idea, since even if it worked, all the Julia work would be done in the main thread.

Regarding your suggestion to provide the list of evaluators to the mclapply, I tried it for a small number of simulations to run in parallel and it seems to work (using htop in the shell you can see the different Julia processes running at the same time). However if there are too many evaluators to create theRJulia(.makeNew = TRUE) command eventually exits with an error because there is not enough available ports.

Thanks again!

johnmchambers commented 6 years ago

I found the bug, I think.

A workaround that allows me to supply the port number is to include the argument startJulia=TRUE in the call to RJulia():

ev = RJulia(port = 1895L, startJulia = TRUE)

ev$port [1] 1895

I'll fix the problem, but need to decide just where.

Thanks for the report

johnmchambers commented 6 years ago

The revision committed as version 0.7.8 should fix this. Will close this issue in a few days if no problems arise with the revised version.

oliviaAB commented 6 years ago

Thank you! The workaround solution works as well, but I will try with the new version.