search

sgearle / bugwas

Locus and lineage tests for bacterial GWAS
GNU General Public License v2.0
30 stars 5 forks source link

bugwas issue when incorporating newick tree #14

Open question25988 opened 1 year ago

question25988 commented 1 year ago

Hello,

I've run into the below error while running DBGWAS. I've skipped step 1 in this case, as there were memory issues that have have been negated by splitting the job into its individual steps. The error seems to be associated with singular value decomposition and the R script? Do you have any idea what's gone wrong please?

Code: run_dbgwas -strains bugs_phenotype_dataset/strains -newick bugs_phenotype_dataset/strains.newick -nc-db Resistance_DB_for_DBGWAS.fasta -pt-db uniprot_sprot_bacteria_for_DBGWAS.fasta -nb-cores 10 -verbose 1 -skip1

Results and error: Step 1. Building DBG and mapping strains on the DBG... Skipping Step 1! build_dbg
map_reads
Done! Step 2. Running statistical test (bugwas + gemma)... Executing Rscript --vanilla /dbgwas/bin/DBGWAS_lib//DBGWAS.R /dbgwas/bin/DBGWASlib/ /ab54/db45/ell/analysis\/dbgwas/phenotype2/output/step1 /ab54/db45/ell/analysis\/dbgwas/phenotype_2/output/step1/bugwas_input.id_phenotype bugwas_out /dbgwas/bin/DBGWASlib//gemma.0.93b 0.01 /ab54/db45/ell/analysis\/dbgwas/phenotype_2/bugs_phenotype_dataset/strains.newick 2>&1... During startup - Warning messages: 1: Setting LC_CTYPE failed, using "C" 2: Setting LC_COLLATE failed, using "C" 3: Setting LC_TIME failed, using "C" 4: Setting LC_MESSAGES failed, using "C" 5: Setting LC_MONETARY failed, using "C" 6: Setting LC_PAPER failed, using "C" 7: Setting LCMEASUREMENT failed, using "C" [DBGWAS] Reading unitigs from /ab54/db45/ell/analysis\/dbgwas/phenotype_2/output/step1/bugwas_input.uniquerows.binary [DBGWAS] Reading phenotypes from /ab54/db45/ell/analysis\/dbgwas/phenotype_2/output/step1/bugwas_input.idphenotype [DBGWAS] Preparing tree for lineage effect analysis from /ab54/db45/ell/analysis\/dbgwas/phenotype_2/bugs_phenotype_dataset/strains.newick [DBGWAS] Restricting genotype 4198013/5594886 patterns with MAF >= 0.01. [DBGWAS] Building kinship matrix Get kinship matrix Reading Files ... -## number of total individuals = 2515 -## number of analyzed individuals = 2515 -## number of covariates = 1 -## number of total SNPs = 4719719 -## number of analyzed SNPs = 4719719 Calculating Relatedness Matrix ... Reading SNPs 0.00% Reading SNPs = 2.12% Reading SNPs == 4.24% Reading SNPs === 6.36% Reading SNPs ==== 8.48% Reading SNPs ===== 10.59% Reading SNPs ====== 12.71% Reading SNPs ======= 14.83% Reading SNPs ======== 16.95% Reading SNPs ========= 19.07% Reading SNPs ========== 21.19% Reading SNPs =========== 23.31% Reading SNPs ============ 25.43% Reading SNPs ============= 27.54% Reading SNPs ============== 29.66% Reading SNPs =============== 31.78% Reading SNPs ================ 33.90% Reading SNPs ================== 36.02% Reading SNPs =================== 38.14% Reading SNPs ==================== 40.26% Reading SNPs ===================== 42.38% Reading SNPs ====================== 44.49% Reading SNPs ======================= 46.61% Reading SNPs ======================== 48.73% Reading SNPs ========================= 50.85% Reading SNPs ========================== 52.97% Reading SNPs =========================== 55.09% Reading SNPs ============================ 57.21% Reading SNPs ============================= 59.33% Reading SNPs ============================== 61.44% Reading SNPs =============================== 63.56% Reading SNPs ================================ 65.68% Reading SNPs ================================= 67.80% Reading SNPs ================================== 69.92% Reading SNPs ==================================== 72.04% Reading SNPs ===================================== 74.16% Reading SNPs ====================================== 76.28% Reading SNPs ======================================= 78.39% Reading SNPs ======================================== 80.51% Reading SNPs ========================================= 82.63% Reading SNPs ========================================== 84.75% Reading SNPs =========================================== 86.87% Reading SNPs ============================================ 88.99% Reading SNPs ============================================= 91.11% Reading SNPs ============================================== 93.23% Reading SNPs =============================================== 95.34% Reading SNPs ================================================ 97.46% Reading SNPs ================================================= 99.58% Reading SNPs ==================================================100.00% [DBGWAS] Performing association tests Rescaled variants. Error in La.svd(x, nu, nv) : BLAS/LAPACK routine 'DLASCL' gave error code -4 Calls: cdbg_lin_loc -> svd -> La.svd Execution halted

[FATAL ERROR] Error executing Rscript --vanilla /dbgwas/bin/DBGWAS_lib//DBGWAS.R /dbgwas/bin/DBGWASlib/ /ab54/db45/ell/analysis\/dbgwas/phenotype2/output/step1 /ab54/db45/ell/analysis\/dbgwas/phenotype_2/output/step1/bugwas_input.id_phenotype bugwas_out /dbgwas/bin/DBGWASlib//gemma.0.93b 0.01 /ab54/db45/ell/analysis\/dbgwas/phenotype_2/bugs_phenotype_dataset/strains.newick 2>&1. Exit status: 256

danny-wilson commented 1 year ago

Hi there,

This is a guess, but it could be a lack-of-memory issue.

Danny

On Mon, 8 May 2023 at 01:41, question25988 @.***> wrote:

Hello,

I've run into the below error while running DBGWAS. I've skipped step 1 in this case, as there were memory issues that have have been negated by splitting the job into its individual steps. The error seems to be associated with singular value decomposition and the R script? Do you have any idea what's gone wrong please?

Code: run_dbgwas -strains bugs_phenotype_dataset/strains -newick bugs_phenotype_dataset/strains.newick -nc-db Resistance_DB_for_DBGWAS.fasta -pt-db uniprot_sprot_bacteria_for_DBGWAS.fasta -nb-cores 10 -verbose 1 -skip1

Results and error: Step 1. Building DBG and mapping strains on the DBG... Skipping Step 1! build_dbg map_reads Done! Step 2. Running statistical test (bugwas + gemma)... Executing Rscript --vanilla /dbgwas/bin/DBGWAS_lib//DBGWAS.R /dbgwas/bin/DBGWASlib/ /ab54/db45/ell/analysis/dbgwas/phenotype2/output/step1 /ab54/db45/ell/analysis/dbgwas/phenotype_2/output/step1/bugwas_input.id_phenotype bugwas_out /dbgwas/bin/DBGWASlib//gemma.0.93b 0.01 /ab54/db45/ell/analysis/dbgwas/phenotype_2/bugs_phenotype_dataset/strains.newick 2>&1... During startup - Warning messages: 1: Setting LC_CTYPE failed, using "C" 2: Setting LC_COLLATE failed, using "C" 3: Setting LC_TIME failed, using "C" 4: Setting LC_MESSAGES failed, using "C" 5: Setting LC_MONETARY failed, using "C" 6: Setting LC_PAPER failed, using "C" 7: Setting LCMEASUREMENT failed, using "C" [DBGWAS] Reading unitigs from /ab54/db45/ell/analysis/dbgwas/phenotype_2/output/step1/bugwas_input.uniquerows.binary [DBGWAS] Reading phenotypes from /ab54/db45/ell/analysis/dbgwas/phenotype_2/output/step1/bugwas_input.idphenotype [DBGWAS] Preparing tree for lineage effect analysis from /ab54/db45/ell/analysis/dbgwas/phenotype_2/bugs_phenotype_dataset/strains.newick [DBGWAS] Restricting genotype 4198013/5594886 patterns with MAF >= 0.01. [DBGWAS] Building kinship matrix Get kinship matrix Reading Files ... -## number of total individuals = 2515 -## number of analyzed individuals = 2515 -## number of covariates = 1 -## number of total SNPs = 4719719 -## number of analyzed SNPs = 4719719 Calculating Relatedness Matrix ... Reading SNPs 0.00% Reading SNPs = 2.12% Reading SNPs == 4.24% Reading SNPs === 6.36% Reading SNPs ==== 8.48% Reading SNPs ===== 10.59% Reading SNPs ====== 12.71% Reading SNPs ======= 14.83% Reading SNPs ======== 16.95% Reading SNPs ========= 19.07% Reading SNPs ========== 21.19% Reading SNPs =========== 23.31% Reading SNPs ============ 25.43% Reading SNPs ============= 27.54% Reading SNPs ============== 29.66% Reading SNPs =============== 31.78% Reading SNPs ================ 33.90% Reading SNPs ================== 36.02% Reading SNPs =================== 38.14% Reading SNPs ==================== 40.26% Reading SNPs ===================== 42.38% Reading SNPs ====================== 44.49% Reading SNPs ======================= 46.61% Reading SNPs ======================== 48.73% Reading SNPs ========================= 50.85% Reading SNPs ========================== 52.97% Reading SNPs =========================== 55.09% Reading SNPs ============================ 57.21% Reading SNPs ============================= 59.33% Reading SNPs ============================== 61.44% Reading SNPs =============================== 63.56% Reading SNPs ================================ 65.68% Reading SNPs ================================= 67.80% Reading SNPs ================================== 69.92% Reading SNPs ==================================== 72.04% Reading SNPs ===================================== 74.16% Reading SNPs ====================================== 76.28% Reading SNPs ======================================= 78.39% Reading SNPs ======================================== 80.51% Reading SNPs ========================================= 82.63% Reading SNPs ========================================== 84.75% Reading SNPs =========================================== 86.87% Reading SNPs ============================================ 88.99% Reading SNPs ============================================= 91.11% Reading SNPs ============================================== 93.23% Reading SNPs =============================================== 95.34% Reading SNPs ================================================ 97.46% Reading SNPs ================================================= 99.58% Reading SNPs ==================================================100.00% [DBGWAS] Performing association tests Rescaled variants. Error in La.svd(x, nu, nv) : BLAS/LAPACK routine 'DLASCL' gave error code -4 Calls: cdbg_lin_loc -> svd -> La.svd Execution halted

[FATAL ERROR] Error executing Rscript --vanilla /dbgwas/bin/DBGWAS_lib//DBGWAS.R /dbgwas/bin/DBGWASlib/ /ab54/db45/ell/analysis/dbgwas/phenotype2/output/step1 /ab54/db45/ell/analysis/dbgwas/phenotype_2/output/step1/bugwas_input.id_phenotype bugwas_out /dbgwas/bin/DBGWASlib//gemma.0.93b 0.01 /ab54/db45/ell/analysis/dbgwas/phenotype_2/bugs_phenotype_dataset/strains.newick 2>&1. Exit status: 256

— Reply to this email directly, view it on GitHub https://github.com/sgearle/bugwas/issues/14, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZJEYABYD3COZJ2QK2N3QDXFA6LRANCNFSM6AAAAAAXZH5J5E . You are receiving this because you are subscribed to this thread.Message ID: @.***>

question25988 commented 1 year ago

Hi Danny, Thanks for your message. Based on checks carried out on the cluster and as as far as I can tell, there doesn't seem to be an issue associated with memory. When the job died, it was using half of the memory assigned to the job.

danny-wilson commented 1 year ago

Ok. The BLAS/LAPACK routine 'DLASCL' gave error code -4 seems to be invalid fourth argument, so perhaps there is an unexpected character that is read in.

It might be possible to debug by adding this to the R code somewhere before the error:

Courtesy of

https://renkun.me/2020/03/31/a-simple-way-to-show-stack-trace-on-error-in-r/

options(error = function() {

calls <- sys.calls()

if (length(calls) >= 2L) {

sink(stderr())

on.exit(sink(NULL))

cat("Backtrace:\n")

calls <- rev(calls[-length(calls)])

for (i in seq_along(calls)) {

  cat(i, ": ", deparse(calls[[i]], nlines = 1L), "\n", sep = "")

}

}

coredump = paste0("coredump.",Sys.getpid(),".RData")

if(coredump!="") save.image(coredump)

if (!interactive()) {

q(status = 1)

}

})

On Tue, 9 May 2023 at 12:50, question25988 @.***> wrote:

Hi Danny, Thanks for your message. Based on checks carried out on the cluster and as as far as I can tell, there doesn't seem to be an issue associated with memory. When the job died, it was using half of the memory assigned to the job.

— Reply to this email directly, view it on GitHub https://github.com/sgearle/bugwas/issues/14#issuecomment-1540010676, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZJEYE5HDDXFX5CPEZEJT3XFIVPNANCNFSM6AAAAAAXZH5J5E . You are receiving this because you commented.Message ID: @.***>