ga parallel does not work when the fitness function is written in Rcpp

[NedaJalali-codes]

Hi,

I want to use ga in parallel format for a function that is written in Rcpp.

When I put parallel = TRUE/parallel = "snow", I got this error:

Error in { : task 1 failed - "NULL value passed as symbol address"

This error usually occurs in "foreach" when we want to call a function from Rcpp in parallel mode. The solution is to write a package that has the Rcpp and then pass the package to the "foreach".

I do not know the same solution is possible in GA.

I appreciate it if you can help me with this issue.

Thanks in advance.

Aden

[luca-scr]

Yes it should work because in the "snow" case all the objects defined and packages loaded in the master process are exported in the worker processes.

[NedaJalali-codes]

Thank you so much for your response. I made a package and load it in R so the error related to Rcpp codes is solved. But I got a new error:

"Error in socketAccept(socket = socket, blocking = TRUE, open = "a+b", : all connections are in use Calls: ... startParallel -> -> makePSOCKcluster -> socketAccept"

Is there any way to debug this error in GA?

Thanks in advance.

Aden

[luca-scr]

It looks like an issue related to parallelisation. Try to create a cluster from scratch and pass that cluster as argument to parallel option in ga() function call. You may want to look at an example in the GA vignette.

[NedaJalali-codes]

Thanks a lot. I will try it.

[NedaJalali-codes]

sorry for retaking your time. Here is the code how I defined the cluster:

nc = detectCores()
print(nc)
cl = makeCluster(nc, type = "PSOCK")
registerDoParallel(cl)
clusterExport(cl, varlist = c("Count_cases_val", "fn_optim_ga", "fix_info", "array_hos"))
clusterCall(cl, library, package = "RcppPackage", character.only = TRUE)

And then I got this error:

Error in socketAccept(socket = socket, blocking = TRUE, open = "a+b", : all connections are in use Calls: makeCluster -> makePSOCKcluster -> socketAccept Execution halted

It is the same error as before.

Aden

[luca-scr]

The message "all connections are in use" seems to suggest an issue with with the parallel backend. Is parallelisation working in general?

[NedaJalali-codes]

Yes. It works.

[NedaJalali-codes]

Hi, I solved the problem and I wanted to share the solution here for others in case they encounter the same issue. I guess the trick was passing the "libpath" to the cluster. ##################### cl <- makePSOCKcluster(8) # I defined cl by this commend registerDoParallel(cl) clusterExport(cl, varlist = c("name of the functions in Rcpp, separated by comma")) clusterEvalQ(cl, .libPaths("~/R")) # pass libpath clusterEvalQ(cl, library(mypackage)) # pass My package which includes Rcpp functions GA_result <- GA::ga(type = "real-valued", fitness = myfun, lower = c(-30,-30), upper = c(0,5), popSize = 100, monitor = FALSE, parallel = cl) stopCluster(cl) print("FinishCL")# for testing that cl works ##################### Best, Aden