privefl / bigsnpr

R package for the analysis of massive SNP arrays.
https://privefl.github.io/bigsnpr/
186 stars 44 forks source link

Node error on HPC when creating LD matrix #372

Closed oalavijeh closed 2 years ago

oalavijeh commented 2 years ago

I work on the 100,000 genomes project under a tight data airlock so apologies for the lack of examples. It is a HPC environment.

When running this loop from your tutorial:

for (chr in 1:22) {

  # print(chr)

  ## indices in 'df_beta'
  ind.chr <- [which](https://rdrr.io/r/base/which.html)(df_beta$chr == chr)
  ## indices in 'G'
  ind.chr2 <- df_beta$`_NUM_ID_`[ind.chr]

  corr0 <- [snp_cor](https://privefl.github.io/bigsnpr/reference/snp_cor.html)(G, ind.col = ind.chr2, size = 3 / 1000,
                   infos.pos = POS2[ind.chr2], ncores = NCORES)

  if (chr == 1) {
    ld <- Matrix::[colSums](https://rdrr.io/pkg/Matrix/man/colSums.html)(corr0^2)
    corr <- [as_SFBM](https://rdrr.io/pkg/bigsparser/man/SFBM-class.html)(corr0, tmp, compact = TRUE)
  } else {
    ld <- [c](https://rdrr.io/r/base/c.html)(ld, Matrix::[colSums](https://rdrr.io/pkg/Matrix/man/colSums.html)(corr0^2))
    corr$add_columns(corr0, [nrow](https://rdrr.io/r/base/nrow.html)(corr))
  }
}

I get:

Error in checkForRemoteErrors(lapply(cl,recvResult))) : one node produced an error: there is no package called 'RHpcBLASctl' 
It is triggered by the line:
corr0 <- [snp_cor](https://privefl.github.io/bigsnpr/reference/snp_cor.html)(G, ind.col = ind.chr2, size = 3 / 1000,
                   infos.pos = POS2[ind.chr2], ncores = NCORES)

I am using an interactive node on the HPC and suspect the other nodes do not have this package installed (I have it installed on the node I am working on). I think I can get around this by writing something like parallel::FUNCTION but I am not sure what that function would be?

Have you had this issue before?

Many thanks

privefl commented 2 years ago

Can't you install the package everywhere? It won't work if you do not have all dependencies everywhere you use the package. I don't understand how it got installed without.

oalavijeh commented 2 years ago

No idea! I've e-mailed the bioinformatics support team and will post an answer here if they can help. Thanks!