how to parallelize "calcGexpCnvBoundaries" and "retestIdentifiedCnvs" ?

[songw01]

Hi there,

Thanks for putting out a nice package for the community!

I am in the middle of proessing ~ 3000 cells to call gene based CNVs. It seems that the pipeline is bottlenecked by the two functions taking up most of the time. I remember reading a thread that you mentioned there is a multi-core capability, but I failed to how to set the parameters. Could you please show me how?

Regards,

[JEFworks]

Hi Song,

Thanks for trying it out!

For processing such a large number of cells, there are a few steps/functions that I anticipate may take a long time. Here are a few tips on how to parallelize those steps to speed up runtime.

1. Identifying potential CNVs

To identify CNVs, HoneyBADGER first uses a HMM model. Currently, you may be running the default pipeline as:

potentialCnvs <- calcGexpCnvBoundaries(gexp.norm, genes, m=0.15, chrs=paste0('chr', c(1:22)), min.traverse=3, t=1e-6, min.num.genes=3, trim=0.1, verbose=FALSE)

This can be parallelized by chromosome, though unfortunately the process is not built in:

# parallelize across 22 cores for 22 chromosomes
potentialCnvs <- mclapply(paste0('chr', c(1:22)), function(chr) {
    calcGexpCnvBoundaries(gexp.norm, genes, m=0.15, chrs=chr, min.traverse=3, t=1e-6, min.num.genes=3, trim=0.1, verbose=FALSE)
}, mc.cores=22)

2. Testing individual CNVs in individual cells

Again, this component can also be parallelized across your identified potential CNVs though unfortunately the parallelization is not built in. For example:

results <- mclapply(potentialCnvs$region, function(region) {
    results <- calcGexpCnvProb(gexp.mats$gexp.norm, gexp.mats$genes, 
     mvFit, region=region, verbose=TRUE)
}, mc.cores=length(potentialCnvs$region))

Because HoneyBADGER leverages information across cells within a Bayesian framework to infer the posterior probability of CNVs within individual cells, I would not recommend parallelizing by splitting up your cells into many small groups for example.

Hope that helps, Jean