parallel computing issues during generate.celltype.data

[krademaker]

Before calculating specificity data for my own dataset, I was following the example code on my sysyem (Ubuntu 18.04, R v3.6.0)

download.file("goo.gl/r5Y24y",
    destfile="expression_mRNA_17-Aug-2014.txt") 
path = "expression_mRNA_17-Aug-2014.txt"
cortex_mrna  = load.linnarsson.sct.data(path)
exp_CortexOnly_DROPPED = drop.uninformative.genes(exp=cortex_mrna$exp,level2annot = cortex_mrna$annot$level2class)
annotLevels = list(level1class=cortex_mrna$annot$level1class,level2class=cortex_mrna$annot$level2class)
fNames_CortexOnly = generate.celltype.data(exp=exp_CortexOnly_DROPPED,annotLevels=annotLevels,groupName="kiCortexOnly")

Which returns the following error:

> fNames_CortexOnly = generate.celltype.data(exp=exp_CortexOnly_DROPPED,annotLevels=annotLevels,groupName="kiCortexOnly")
Loading required package: parallel
Error in makePSOCKcluster(names = spec, ...) : 
  numeric 'names' must be >= 1

Apparently, determining the n.o. cores (source code) returns a NA, which in turn causes makeCluster() to fail. On what OS was the code developed / tested? Perhaps this is a Ubuntu specific issue?

[NathanSkene]

It was developed on mac. Pretty sure I've run it on a Unix cluster as well. Check if there are known Ubuntu issues with the parallel package? If so, easiest way is probably to add a test for whether OS is Ubuntu and convert mclapply calls to lapply. Let me know how it goes.

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: Koen Rademaker notifications@github.com Sent: Tuesday, May 28, 2019 9:42:58 PM To: NathanSkene/EWCE Cc: Subscribed Subject: [NathanSkene/EWCE] parallel computing issues during generate.celltype.data (#5)

Before calculating specificity data for my own dataset, I was following the example code on my sysyem (Ubuntu 18.04, R v3.6.0)

download.file("goo.gl/r5Y24y", destfile="expression_mRNA_17-Aug-2014.txt") path = "expression_mRNA_17-Aug-2014.txt" cortex_mrna = load.linnarsson.sct.data(path) exp_CortexOnly_DROPPED = drop.uninformative.genes(exp=cortex_mrna$exp,level2annot = cortex_mrna$annot$level2class) annotLevels = list(level1class=cortex_mrna$annot$level1class,level2class=cortex_mrna$annot$level2class) fNames_CortexOnly = generate.celltype.data(exp=exp_CortexOnly_DROPPED,annotLevels=annotLevels,groupName="kiCortexOnly")

Which returns the following error:

fNames_CortexOnly = generate.celltype.data(exp=exp_CortexOnly_DROPPED,annotLevels=annotLevels,groupName="kiCortexOnly") Loading required package: parallel Error in makePSOCKcluster(names = spec, ...) : numeric 'names' must be >= 1

Apparently, determining the n.o. cores (source codehttps://github.com/NathanSkene/EWCE/blob/a4e4e4595eb016321094b814a78fc01c1674103c/R/generate.celltype.data.r#L28) returns a NA, which in turn causes makeCluster() to fail. On what OS was the code developed / tested? Perhaps this is a Ubuntu specific issue?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/NathanSkene/EWCE/issues/5?email_source=notifications&email_token=AH5ZPEYR4FNCWBYKWL3JWH3PXWKNDA5CNFSM4HQG54T2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GWKK3EQ, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH5ZPE6ZORJRPXAWPBLQGVTPXWKNDANCNFSM4HQG54TQ.

[krademaker]

Hi,

so I repeated running the code a few times, and sporadically detectCores returns NA, while in other instances it returns 8 (matching the n.o. cores on my system), after which I can successfully run generate.celltype.data.

I am not sure why detecting the number of cores fails every now and then, but the original issue is resolved for me right now.