tgen / CovGen

Creates a target specific exome_full192.coverage.txt file required by MutSig
MIT License
21 stars 9 forks source link

Parallelized SnpEff causes race condition in downloading databases #14

Closed ning-y closed 3 years ago

ning-y commented 3 years ago

If the specified SnpEff database version has not yet been downloaded, the parallelized SnpEff instances called by the bash script will cause three simultaneous download attempts into the same /tmp/ subdirectory (because SnpEff automatically downloads databases if they are not already downloaded, and the /tmp/ subdirectory used to cache the download is hard-coded), probably setting up some race condition, and causing all the downloads to fail:

java.lang.RuntimeException: java.io.FileNotFoundException: /tmp/snpEff_v5_0_GRCh37.87.zip (No such file or directory)
        at org.snpeff.util.Download.unzip(Download.java:395)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdDownload.downloadAndInstall(SnpEffCmdDownload.java:33)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdDownload.runDownloadGenome(SnpEffCmdDownload.java:86)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdDownload.run(SnpEffCmdDownload.java:72)
        at org.snpeff.SnpEff.run(SnpEff.java:1227)
        at org.snpeff.SnpEff.loadDb(SnpEff.java:514)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:940)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:923)
        at org.snpeff.SnpEff.run(SnpEff.java:1188)
        at org.snpeff.SnpEff.main(SnpEff.java:168)
Caused by: java.io.FileNotFoundException: /tmp/snpEff_v5_0_GRCh37.87.zip (No such file or directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
        at org.snpeff.util.Download.unzip(Download.java:330)
        ... 9 more
java.lang.RuntimeException: Genome download failed!
        at org.snpeff.SnpEff.loadDb(SnpEff.java:515)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:940)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:923)
        at org.snpeff.SnpEff.run(SnpEff.java:1188)
        at org.snpeff.SnpEff.main(SnpEff.java:168)

I do not think there is an easy fix to this, so I would propose adding a disclaimer for this in the README.