genetics-of-dna-methylation-consortium / godmc_phase2

This repository contains the code to run the analysis pipeline for phase 2 of goDMC released June 2024.
GNU General Public License v3.0
2 stars 1 forks source link

[Bug]: 09 - Error in psoriasisNOHLA EWAS #34

Closed feichel closed 1 month ago

feichel commented 1 month ago

Contact Details

fabian.eichelmann@dife.de

Scripts

09-PRS_EWAS.sh

What happened?

I encounter an error message related to isva when running EWAS on psoriasisNOHLA. The other PGS worked fine...

How can the bug be reproduced?

No response

R version

4.3.0 (April, 2023)

Python version

None

Relevant log output

Loading PRS data for PsoriasisNOHLA

Performing EWAS for PsoriasisNOHLA on 1320 individuals and 837405 CpGs
Error in La.svd(V) : error code 1 from Lapack routine 'dgesdd'
Calls: main ... add.surrogate.variables -> isva -> isvaFn -> fastICA -> La.svd
Execution halted
msolerartigas commented 1 month ago

Hi Fabian,

Thanks for your message!

The error you are getting seems to be related to the Singular Value Decomposition (SVD) function (La.svd) from the LAPACK library. From what I see this error could be related to the methylation matrix that is feed into the analysis and it could also be that the analysis is running out of memory. Given that the analyses for the other PRS, using the exact same matrix have worked, I do not think the issue relates to the matrix. I was wondering whether you may be running the analyses is a computer cluster setting and at that point for some reason there may have been a memory problem. Could you perhaps try to run it again making sure there is enough memory and see what happens?

Many thanks for running these analyses!

María

feichel commented 1 month ago

Hi María,

thank you for the suggestion. The memory was not the issue, but I found a workaround for the error.

I switched out the LAPACK library using the following code on our hpc


sudo update-alternatives --config libblas.so.3-x86_64-linux-gnu
There are 2 choices for the alternative libblas.so.3-x86_64-linux-gnu (providing /usr/lib/x86_64-linux-gnu/libblas.so.3).

  Selection    Path                                          Priority   Status
------------------------------------------------------------
  0            /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3   35        auto mode
  1            /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3   35        manual mode
* 2            /usr/lib/x86_64-linux-gnu/blas/libblas.so.3    10        manual mode

Press <enter> to keep the current choice[*], or type selection number:
sudo update-alternatives --config liblapack.so.3-x86_64-linux-gnu
There are 2 choices for the alternative liblapack.so.3-x86_64-linux-gnu (providing /usr/lib/x86_64-linux-gnu/liblapack.so.3).

  Selection    Path                                             Priority   Status
------------------------------------------------------------
  0            /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3    35        auto mode
  1            /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3    35        manual mode
* 2            /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3   10        manual mode

Press <enter> to keep the current choice[*], or type selection number:

Before

R version 4.3.0 (2023-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.5 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3 
LAPACK: /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3.10.3;  LAPACK version 3.7.1

After

sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.5 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

Runtime was longer, but it finished without error and the results look ok to me. I suspect that updating our R/Ubuntu versions would have also done the trick, but that was not something I wanted to try yet. I found some background info on LAPACK and how to switch the defaults here: Link

I addition, I found an error in check_upload.sh (or 09-PRS_EWAS) that mistakenly leads to a failed check. The final check for module 9 in check_upload looks for "script finalised" whereas the logs in 09 print "script finalized" =).