gbm 2.x clashes with mclapply

[GoogleCodeExporter]

What steps will reproduce the problem?
1.

Add these two lines to .Rprofile

cat(paste("This session PID is ", system("echo $PPID",intern=TRUE), ":\n",sep = 
""))
cat(paste("begun at ", as.POSIXct(date(), format = "%a %b %d %H:%M:%S %Y"), 
":\n",sep = ""))

2.
Source the attached file to create a function which uses the gbm function 
example code into a function to use with mclapply using different seed numbers.

3.

What is the expected output? What do you see instead?

Using gbm-1.6-3.1, I get this as expected;

> testing(4)
  2013-08-21 20:57:31  Begin using multicore method with phony data with 4 cores.
Core 1 uses 20442 

Core 2 uses 20443 

Core 3 uses 20445 

Core 4 uses 20447 

 2013-08-21 20:57:36 
....Completed testing multicore method with invented data.
$a
   CV Test OOB
1 126  131  79

$b
   CV Test OOB
1 117  121  85

$c
   CV Test OOB
1 157  126  83

$d
   CV Test OOB
1 123  140  81

Using the current gbm package, I get this:

system.time(bbb <- testing(4))
  2013-08-21 16:18:03  Begin using multicore method with phony data with 4 cores.

Core 1 uses 22812 

Core 2 uses 22814 

Core 3 uses 22816 

Core 4 uses 22819 
This session PID is 22821:
begun at 2013-08-21 16:18:04:
This session PID is 22829:
begun at 2013-08-21 16:18:04:
This session PID is 22838:
begun at 2013-08-21 16:18:04:
This session PID is 22847:
begun at 2013-08-21 16:18:04:
 2013-08-21 16:18:07 
....Completed testing multicore method with invented data.
   user  system elapsed 
  0.460   1.760   3.926 
Warning message:
In mclapply(subsets, FUN = test.gbm, mc.cores = nc, mc.cleanup = FALSE,  :
  3 function calls resulted in an error

bbb$b
[1] "Error in socketConnection(\"localhost\", port = port, server = TRUE, 
blocking = TRUE,  : \n  cannot open the connection\n"
attr(,"class")
[1] "try-error"
attr(,"condition")
<simpleError in socketConnection("localhost", port = port, server = TRUE, 
blocking = TRUE,     open = "a+b", timeout = timeout): cannot open the 
connection>
> 

bbb$a works properly and the errors on bbb$c and bbb$d are identical
to the above.

Notice as well as starting a new R process for each parallel process, another 
one is immediately stared within each one.  That prevents mclapply from 
collecting more than the first element of subsets.

What version of the product are you using? On what operating system?

To work, it needs gbm-1.6-3.1, otherwise gbm-2.x gives the result above.  The 
old version is on a Fedora 15 (32bit) installation, the newer one on a Kubuntu 
10.04 (64bit)

Please provide any additional information below.

When more challenging functions are tried, instead of just one extra R process 
being started, one for each core on the machine (irrespective of how many are 
asked for) are started.

Setting mc.cores to 1 will work 9i.e. return the complete list) but it still 
starts those extra R processes and it takes twice as long as on an identical 
hardware running Windows 7.

Original issue reported on code.google.com by duck...@gmail.com on 23 Aug 2013 at 9:03

Attachments:

• testing.fn.sc

[GoogleCodeExporter]

Thanks for reporting this.

You report using gbm 2.x. Please be more specific.

I've just run the code using a version of gbm that will probablyy be released 
onto the project page later today and I don't get the error that you do. I 
would have thought that 2.0-0.1 would also work.

If ncores=1, gbmCluster just returns NULL and no attempt is made at shutting 
down the cluster on exit. That was introduced in 2.0-0.1 (so far as my memory 
can be relied on).

Thanks again,
Harry

Original comment by harry.southworth on 27 Aug 2013 at 9:27

[GoogleCodeExporter]

I meant 2.1-0.1, not 2.0

Original comment by harry.southworth on 27 Aug 2013 at 9:58

[GoogleCodeExporter]

Sorry, I can't be more specific.  It's probably the one that has the 
publication date of 2013-05-10 which is described only as 2.1.  The archives 
list only gbm_2.0-5 and gbm_2.0-8 which I rashly assumed would have the problem 
that I encountered with 2.1. 2.1-0.1 might indeed not have that same issue.

Thanks for looking into it.

Original comment by duck...@gmail.com on 27 Aug 2013 at 11:22