TheJacksonLaboratory / LIRICAL

LIkelihood Ratio Interpretation of Clinical AbnormaLities
https://thejacksonlaboratory.github.io/LIRICAL/stable
Other
24 stars 11 forks source link

Database download error! #660

Open MiladEidi opened 4 months ago

MiladEidi commented 4 months ago

Hi dear developers, when I try to download lirical's database it returns these:

(base) milad@DESKTOP-VI9DTD8:/mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2$ java -jar lirical-cli-2.0.2.jar D SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider]


| | | | | _/ | / \ | | | | | || |) || | | / \ | | | | | || _ < | | | / | |_ |___||| ___// ____|

LIkelihood Ratio Interpretation of Clinical AbnormaLities :: v2.0.2 ::

08:44:11.712 [main] INFO o.m.lirical.cli.cmd.DownloadCommand - Downloading data to /mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2/data 08:44:12.221 [main] INFO o.m.biodownload.BioDownloaderBuilder - The builder validation is successful. 08:44:12.222 [main] INFO o.m.biodownload.BioDownloaderImpl - Cowardly refusing to download "hp.json" since we found it at "/mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2/data/hp.json" 08:44:12.223 [main] INFO o.m.biodownload.BioDownloaderImpl - Cowardly refusing to download "phenotype.hpoa" since we found it at "/mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2/data/phenotype.hpoa" 08:44:12.223 [main] INFO o.m.biodownload.BioDownloaderImpl - Cowardly refusing to download "hgnc_complete_set.txt" since we found it at "/mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2/data/hgnc_complete_set.txt" 08:44:12.223 [main] INFO o.m.biodownload.BioDownloaderImpl - Cowardly refusing to download "mim2gene_medgen" since we found it at "/mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2/data/mim2gene_medgen" 08:44:12.912 [main] ERROR o.m.biodownload.FileDownloader - Failed to downloaded file from storage.googleapis.com java.io.IOException: Server returned HTTP response code: 403 for URL: https://storage.googleapis.com/ielis/jannovar/v0.35/hg19_ucsc.ser at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484) at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:2044) at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:2039) at java.base/java.security.AccessController.doPrivileged(AccessController.java:569) at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:2038) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1605) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1585) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224) at org.monarchinitiative.biodownload.FileDownloader.copyURLToFileThroughURL(FileDownloader.java:106) at org.monarchinitiative.biodownload.FileDownloader.copyURLToFile(FileDownloader.java:86) at org.monarchinitiative.biodownload.BioDownloaderImpl.downloadFileIfNeeded(BioDownloaderImpl.java:69) at org.monarchinitiative.biodownload.BioDownloaderImpl.download(BioDownloaderImpl.java:49) at org.monarchinitiative.lirical.cli.cmd.DownloadCommand.execute(DownloadCommand.java:57) at org.monarchinitiative.lirical.cli.cmd.BaseCommand.call(BaseCommand.java:31) at org.monarchinitiative.lirical.cli.cmd.DownloadCommand.call(DownloadCommand.java:21) at org.monarchinitiative.lirical.cli.cmd.BaseCommand.call(BaseCommand.java:18) at picocli.CommandLine.executeUserObject(CommandLine.java:2041) at picocli.CommandLine.access$1500(CommandLine.java:148) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461) at picocli.CommandLine$RunLast.handle(CommandLine.java:2453) at picocli.CommandLine$RunLast.handle(CommandLine.java:2415) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273) at picocli.CommandLine$RunLast.execute(CommandLine.java:2417) at picocli.CommandLine.execute(CommandLine.java:2170) at org.monarchinitiative.lirical.cli.Main.main(Main.java:46) Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: https://storage.googleapis.com/ielis/jannovar/v0.35/hg19_ucsc.ser at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1993) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1585) at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) at org.monarchinitiative.biodownload.FileDownloader.connect(FileDownloader.java:149) at org.monarchinitiative.biodownload.FileDownloader.copyURLToFileThroughURL(FileDownloader.java:104) ... 16 common frames omitted 08:44:12.920 [main] ERROR o.m.biodownload.BioDownloaderImpl - Error downloading "hg19_ucsc.ser" from "https://storage.googleapis.com/ielis/jannovar/v0.35/hg19_ucsc.ser": ERROR: Problem downloading file: Server returned HTTP response code: 403 for URL: https://storage.googleapis.com/ielis/jannovar/v0.35/hg19_ucsc.ser 08:44:12.920 [main] ERROR o.m.lirical.cli.cmd.DownloadCommand - Error: ERROR: Problem downloading file: Server returned HTTP response code: 403 for URL: https://storage.googleapis.com/ielis/jannovar/v0.35/hg19_ucsc.ser org.monarchinitiative.biodownload.FileDownloadException: ERROR: Problem downloading file: Server returned HTTP response code: 403 for URL: https://storage.googleapis.com/ielis/jannovar/v0.35/hg19_ucsc.ser at org.monarchinitiative.biodownload.FileDownloader.copyURLToFileThroughURL(FileDownloader.java:136) at org.monarchinitiative.biodownload.FileDownloader.copyURLToFile(FileDownloader.java:86) at org.monarchinitiative.biodownload.BioDownloaderImpl.downloadFileIfNeeded(BioDownloaderImpl.java:69) at org.monarchinitiative.biodownload.BioDownloaderImpl.download(BioDownloaderImpl.java:49) at org.monarchinitiative.lirical.cli.cmd.DownloadCommand.execute(DownloadCommand.java:57) at org.monarchinitiative.lirical.cli.cmd.BaseCommand.call(BaseCommand.java:31) at org.monarchinitiative.lirical.cli.cmd.DownloadCommand.call(DownloadCommand.java:21) at org.monarchinitiative.lirical.cli.cmd.BaseCommand.call(BaseCommand.java:18) at picocli.CommandLine.executeUserObject(CommandLine.java:2041) at picocli.CommandLine.access$1500(CommandLine.java:148) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461) at picocli.CommandLine$RunLast.handle(CommandLine.java:2453) at picocli.CommandLine$RunLast.handle(CommandLine.java:2415) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273) at picocli.CommandLine$RunLast.execute(CommandLine.java:2417) at picocli.CommandLine.execute(CommandLine.java:2170) at org.monarchinitiative.lirical.cli.Main.main(Main.java:46)

I tried it several times in different days, but the error still lasts... please help!

ielis commented 4 months ago

Hi @MiladEidi

Running

wget https://storage.googleapis.com/ielis/jannovar/v0.35/hg19_ucsc.ser

runs OK on my end, so I do not think there are any issues with the hosting. Your connectivity issues may be a result being behind a firewall (or similar), and, unfortunately, it is impossible for me to help you with that.

As a workaround, you can download the .ser files on another machine, and then copy them into the /mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2/data folder.

Please have a look here, there are 4 URLs that are hosted on Google Cloud storage.

This is how the data folder should look like at the end:

data
├── hg19_refseq.ser
├── hg19_ucsc.ser
├── hg38_refseq.ser
├── hg38_ucsc.ser
├── hgnc_complete_set.txt
├── hp.json
├── mim2gene_medgen
└── phenotype.hpoa

I'd be happy to help further if the above does not work, please let me know.

MiladEidi commented 4 months ago

Thank you bro, but I downloaded the databases somewhere else... Now I am getting another error as follows:

(base) milad@DESKTOP-VI9DTD8:/mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2$ java -jar lirical-cli-2.0.2.jar R -p=HP:0000821,HP:0000851 -d=/mnt/c/Users/Milad/OneDrive/Desktop/lirical_dbs/ --use-orphanet --exomiser-hg38=/mnt/e/NGSneeds/Exomiser/hg38_exomiser/2402_hg38/2402_hg38_variants.mv.db --vcf=/mnt/c/Users/Milad/OneDrive/Desktop/CH-063-04.hard-filtered.vcf.gz --assembly=hg38 -o=/mnt/c/Users/Milad/OneDrive/Desktop/CH-063-04 SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider]


| | | | | _/ | / \ | | | | | || |) || | | / \ | | | | | || _ < | | | / | |_ |___||| ___// ____|

LIkelihood Ratio Interpretation of Clinical AbnormaLities :: v2.0.2 ::

16:06:31.232 [main] INFO o.m.l.c.c.LiricalConfigurationCommand - Running LIRICAL from /mnt/c/Users/Milad/OneDrive/Desktop/lirical-cli-2.0.2/lirical-cli-2.0.2.jar 16:06:31.234 [main] INFO o.m.l.c.c.LiricalConfigurationCommand - Using data folder at /mnt/c/Users/Milad/OneDrive/Desktop/lirical_dbs 16:06:31.234 [main] INFO o.m.l.c.c.AbstractPrioritizeCommand - Parsing the analysis inputs 16:06:31.235 [main] INFO o.m.l.c.c.AbstractPrioritizeCommand - Bootstrapping LIRICAL 16:06:34.363 [main] INFO o.m.l.c.c.AbstractPrioritizeCommand - Configured LIRICAL v2.0.2 16:06:34.367 [main] INFO o.m.l.c.c.AbstractPrioritizeCommand - Input sanitation found no issues 16:06:34.375 [main] INFO o.m.l.i.s.JannovarFunctionalVariantAnnotatorService - Loading transcript database from /mnt/c/Users/Milad/OneDrive/Desktop/lirical_dbs/hg38_refseq.ser java.lang.IllegalStateException: The read format 3 is larger than the supported format 1 [1.4.200/5] at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:950) at org.h2.mvstore.MVStore.readStoreHeader(MVStore.java:750) at org.h2.mvstore.MVStore.(MVStore.java:396) at org.h2.mvstore.MVStore$Builder.open(MVStore.java:3579) at org.monarchinitiative.lirical.exomiser_db_adapter.ExomiserMvStoreMetadataService.of(ExomiserMvStoreMetadataService.java:48) at org.monarchinitiative.lirical.exomiser_db_adapter.ExomiserMvStoreMetadataServiceFactory.getVariantMetadataService(ExomiserMvStoreMetadataServiceFactory.java:28) at org.monarchinitiative.lirical.io.vcf.VcfVariantParserFactory.forPath(VcfVariantParserFactory.java:42) at org.monarchinitiative.lirical.cli.cmd.LiricalConfigurationCommand.readVariantsFromVcfFile(LiricalConfigurationCommand.java:276) at org.monarchinitiative.lirical.cli.cmd.AbstractPrioritizeCommand.prepareAnalysisData(AbstractPrioritizeCommand.java:158) at org.monarchinitiative.lirical.cli.cmd.AbstractPrioritizeCommand.execute(AbstractPrioritizeCommand.java:97) at org.monarchinitiative.lirical.cli.cmd.PrioritizeCommand.execute(PrioritizeCommand.java:10) at org.monarchinitiative.lirical.cli.cmd.BaseCommand.call(BaseCommand.java:31) at org.monarchinitiative.lirical.cli.cmd.OutputCommand.call(OutputCommand.java:13) at org.monarchinitiative.lirical.cli.cmd.BaseCommand.call(BaseCommand.java:18) at picocli.CommandLine.executeUserObject(CommandLine.java:2041) at picocli.CommandLine.access$1500(CommandLine.java:148) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461) at picocli.CommandLine$RunLast.handle(CommandLine.java:2453) at picocli.CommandLine$RunLast.handle(CommandLine.java:2415) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273) at picocli.CommandLine$RunLast.execute(CommandLine.java:2417) at picocli.CommandLine.execute(CommandLine.java:2170) at org.monarchinitiative.lirical.cli.Main.main(Main.java:46)

do you where is the problem?

MiladEidi commented 4 months ago

I think there is a bug when I want to use LIRICAL with hg38 version of human genome. the Exomiser database I am using is "2402_hg38_variants.mv.db".

Please help... Best, Milad

ielis commented 4 months ago

Hi @MiladEidi this is because LIRICAL does not yet support the latest Exomiser variant database format.

The latest data release supported by LIRICAL is 2309.

We plan adding the support for the latest format.

MiladEidi commented 4 months ago

uh, thank you bro for the prompt support