Running numbat failed on large datasets: cannot allocate memory block of size 134217728 Tb

[JikaiShao]

Hi!

Thanks again for developing this tools!

I have successfully ran numbat on many tumor samples, including several large samples (20,000 - 30,000 cell) by reducing the number of cores (thanks for your suggestions!). However, I failed to run numbat on two extremely large sample (more than 50,000 cells) though I have set the 'ncores' and 'ncores_nni' to 4 and ran numbat on a workstation with more than 1 TB RAM.

Here is the output:

Running under parameters:
t = 1e-05
alpha = 1e-04
gamma = 20
min_cells = 50
init_k = 4
max_cost = 17270.7
max_iter = 2
max_nni = 100
min_depth = 0
use_loh = auto
multi_allelic = TRUE
min_LLR = 5
min_overlap = 0.45
max_entropy = 0.5
skip_nj = FALSE
diploid_chroms = 
ncores = 4
ncores_nni = 4
common_diploid = TRUE
tau = 0.3
check_convergence = TRUE
plot = TRUE
genome = hg38
Input metrics:
57569 cells
Mem used: 18.7Gb
Approximating initial clusters using smoothed expression ..
Mem used: 18.7Gb
number of genes left: 11554
running hclust...
Iteration 1
Mem used: 65.1Gb
Running HMMs on 7 cell groups..
Retesting CNVs..
Retesting CNVs..
Retesting CNVs..
Retesting CNVs..
Retesting CNVs..
Retesting CNVs..
Retesting CNVs..
Expression noise level: medium (0.8). 
Running HMMs on 4 cell groups..
Testing for multi-allelic CNVs ..
0 multi-allelic CNVs found: 
Evaluating CNV per cell ..
Mem used: 28.1Gb
All cells succeeded
Expanding allelic states..
No multi-allelic CNVs, skipping ..
No multi-allelic CNVs, skipping ..
No multi-allelic CNVs, skipping ..
Building phylogeny ..
Mem used: 31Gb
Using 11 CNVs to construct phylogeny
Error in ape::nj(dist_mat) : 
  cannot allocate memory block of size 134217728 Tb
Calls: run_numbat ... reorder -> <Anonymous> -> <Anonymous> -> <Anonymous>
In addition: Warning messages:
1: In asMethod(object) :
  sparse->dense coercion: allocating vector of size 6.0 GiB
2: In asMethod(object) :
  sparse->dense coercion: allocating vector of size 5.0 GiB
3: In asMethod(object) :
  sparse->dense coercion: allocating vector of size 6.4 GiB
4: In asMethod(object) :
  sparse->dense coercion: allocating vector of size 6.0 GiB
Execution halted

Currently I am trying to split the input DGE into several subsets (each with 10,000 to 20,000 cell) and then re-run numbat on each subset. I was wondering if you would be willing to give me some suggestions on this.

Thanks, Jack

[teng-gao]

Hi Jack,

50k+ cells is a record :) Looking at the error I think the input size broke ape::nj. You can try set skip_nj = TRUE to skip neighbor-joining tree and always use UPGMA to initialize phylogeny.

Best, Teng

[JikaiShao]

Hi Teng,

Thanks for your suggestions! I have run it using skip_nj = TRUE and hope it will successfully go into the 2nd iteration :). Is there any potential difference between the default output and the output with the skipping of neighbor-joining tree construction?

Thank you very much!

Yours, Jack

[teng-gao]

NJ/UPGMA are two alternative ways to initialize the maximum likelihood phylogeny search via NNI. The default tries both and uses the tree with higher likelihood as initial tree. The end result phylogeny may differ with different starting points, but should be similar in most cases. For large datasets (>10k cells), we recommend only using UPGMA because it is faster than NJ, and NJ doesn't necessarily yield better tree.