search

wshuai294 / PStrain

Pstrain profiles strains in metagenomics data. It infers strain abundance and genotype for each species. Also, it has a single species mode; where given a BAM and VCF, it can phase the variants for any species.
MIT License
8 stars 2 forks source link

Could not build fai index ./output/test//ref//merged_ref.fa.fai #4

Closed weiting-liang closed 3 years ago

weiting-liang commented 3 years ago

Hi shuai! I ran "test", but it showed following. What should I do?

Database is loaded. ############## Could not build fai index ./output/test//ref//merged_ref.fa.fai Settings: Output files: "./output/test//ref//merged_ref..bt2" Line rate: 6 (line is 64 bytes) Lines per side: 1 (side is 64 bytes) Offset rate: 4 (one in 16) FTable chars: 10 Strings: unpacked Max bucket size: default Max bucket size, sqrt multiplier: default Max bucket size, len divisor: 4 Difference-cover sample period: 1024 Endianness: little Actual local endianness: little Sanity checking: disabled Assertions: disabled Random seed: 0 Sizeofs: void:8, int:4, long:8, size_t:8 Input files DNA, FASTA: ./output/test//ref//merged_ref.fa Warning: Empty fasta file: './output/test//ref//merged_ref.fa' Warning: All fasta inputs were empty Total time for call to driver() for forward index: 00:00:00 Error: Encountered internal Bowtie 2 exception (#1) Command: bowtie2-build --wrapper basic-0 -f ./output/test//ref//merged_ref.fa ./output/test//ref//merged_ref Could not locate a Bowtie index corresponding to basename "./output/test//ref//merged_ref" Error: Encountered internal Bowtie 2 exception (#1) Command: /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/../install/packages/bowtie2-2.3.1-legacy/bowtie2-align-s --wrapper basic-0 -x ./output/test//ref//merged_ref -p 1 -1 /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/test_1.fq.gz -2 /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/test_2.fq.gz (ERR): bowtie2-align exited with value 1 [Fri Mar 12 16:13:32 CST 2021] picard.sam.AddOrReplaceReadGroups INPUT=./output/test//map/mapped.sort.bam OUTPUT=./output/test/map/mapped.bam RGLB=whatever RGPL=illumina RGPU=whatever RGSM=whatever RGID=1 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json [Fri Mar 12 16:13:32 CST 2021] Executing as liangweiting@cngb-oxcompute-9.cngb.sz.hpc on Linux 2.6.32-696.30.1.el6.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_265-b11; Picard version: 2.1.0(25ebc07f7fbaa7c1a4a8e6c130c88c1d10681802_1454776546) JdkDeflater INFO 2021-03-12 16:13:32 AddOrReplaceReadGroups Created read group ID=1 PL=illumina LB=whatever SM=whatever

[Fri Mar 12 16:13:32 CST 2021] picard.sam.AddOrReplaceReadGroups done. Elapsed time: 0.00 minutes. Runtime.totalMemory()=2027945984 INFO 16:13:38,458 HelpFormatter - -------------------------------------------------------------------------------- INFO 16:13:38,462 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.5-0-g36282e4, Compiled 2015/11/25 04:03:56 INFO 16:13:38,462 HelpFormatter - Copyright (c) 2010 The Broad Institute INFO 16:13:38,463 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk INFO 16:13:38,468 HelpFormatter - Program Args: -T HaplotypeCaller -R ./output/test//ref//merged_ref.fa -allowPotentiallyMisencodedQuals -I ./output/test//map/mapped.bam -o ./output/test//map/mapped.vcf.gz INFO 16:13:38,479 HelpFormatter - Executing as liangweiting@cngb-oxcompute-9.cngb.sz.hpc on Linux 2.6.32-696.30.1.el6.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_265-b11. INFO 16:13:38,480 HelpFormatter - Date/Time: 2021/03/12 16:13:38 INFO 16:13:38,480 HelpFormatter - -------------------------------------------------------------------------------- INFO 16:13:38,480 HelpFormatter - -------------------------------------------------------------------------------- INFO 16:13:38,595 GenomeAnalysisEngine - Strictness is SILENT INFO 16:13:39,470 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,470 HttpMethodDirector - Retrying request INFO 16:13:39,472 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,473 HttpMethodDirector - Retrying request INFO 16:13:39,474 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,475 HttpMethodDirector - Retrying request INFO 16:13:39,476 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,476 HttpMethodDirector - Retrying request INFO 16:13:39,478 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,478 HttpMethodDirector - Retrying request

ERROR ------------------------------------------------------------------------------------------
ERROR A USER ERROR has occurred (version 3.5-0-g36282e4):
ERROR
ERROR This means that one or more arguments or inputs in your command are incorrect.
ERROR The error message below tells you what is the problem.
ERROR
ERROR If the problem is an invalid argument, please check the online documentation guide
ERROR (or rerun your command with --help) to view allowable command-line arguments for this tool.
ERROR
ERROR Visit our website and forum for extensive documentation and answers to
ERROR commonly asked questions http://www.broadinstitute.org/gatk
ERROR
ERROR Please do NOT post this error to the GATK forum unless you have really tried to fix it yourself.
ERROR
ERROR MESSAGE: Fasta index file /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/./output/test/ref/merged_ref.fa.fai for reference /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/./output/test/ref/merged_ref.fa does not exist. Please see http://gatkforums.broadinstitute.org/discussion/1601/how-can-i-prepare-a-fasta-file-to-use-as-reference for help creating it.
ERROR ------------------------------------------------------------------------------------------

...................................... [E::hts_open_format] Failed to open file "./output/test//map/mapped.vcf.gz" : No such file or directory No need to merge in case of one sample.

Sincerely, Liang

wshuai294 commented 3 years ago

Maybe your Metaphlan2 didn't work, is there any file in ./output/test//metaphlan2 folder? if it works, it will contain a file metaphlan2_output.txt. You can install a Metaphlan2 yourself, and appoint its path by the parameter --metaphlan2.

Please let me know if it helps.

Thank you. WANG Shuai

On Fri, Mar 12, 2021 at 4:16 PM liangweiting-student < @.***> wrote:

I ran "test", but it showed like following:

Database is loaded. ############## Could not build fai index ./output/test//ref//merged_ref.fa.fai Settings: Output files: "./output/test//ref//merged_ref.

.bt2" Line rate: 6 (line is 64 bytes) Lines per side: 1 (side is 64 bytes) Offset rate: 4 (one in 16) FTable chars: 10 Strings: unpacked Max bucket size: default Max bucket size, sqrt multiplier: default Max bucket size, len divisor: 4 Difference-cover sample period: 1024 Endianness: little Actual local endianness: little Sanity checking: disabled Assertions: disabled Random seed: 0 Sizeofs: void:8, int:4, long:8, size_t:8 Input files DNA, FASTA: ./output/test//ref//merged_ref.fa Warning: Empty fasta file: './output/test//ref//merged_ref.fa' Warning: All fasta inputs were empty Total time for call to driver() for forward index: 00:00:00 Error: Encountered internal Bowtie 2 exception (#1 https://github.com/wshuai294/PStrain/issues/1) Command: bowtie2-build --wrapper basic-0 -f ./output/test//ref//merged_ref.fa ./output/test//ref//merged_ref Could not locate a Bowtie index corresponding to basename "./output/test//ref//merged_ref" Error: Encountered internal Bowtie 2 exception (#1 https://github.com/wshuai294/PStrain/issues/1) Command: /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/../install/packages/bowtie2-2.3.1-legacy/bowtie2-align-s --wrapper basic-0 -x ./output/test//ref//merged_ref -p 1 -1 /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/test_1.fq.gz -2 /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/test_2.fq.gz (ERR): bowtie2-align exited with value 1 [Fri Mar 12 16:13:32 CST 2021] picard.sam.AddOrReplaceReadGroups INPUT=./output/test//map/mapped.sort.bam OUTPUT=./output/test/map/mapped.bam RGLB=whatever RGPL=illumina RGPU=whatever RGSM=whatever RGID=1 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json [Fri Mar 12 16:13:32 CST 2021] Executing as @.*** on Linux 2.6.32-696.30.1.el6.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_265-b11; Picard version: 2.1.0(25ebc07f7fbaa7c1a4a8e6c130c88c1d10681802_1454776546) JdkDeflater INFO 2021-03-12 16:13:32 AddOrReplaceReadGroups Created read group ID=1 PL=illumina LB=whatever SM=whatever

[Fri Mar 12 16:13:32 CST 2021] picard.sam.AddOrReplaceReadGroups done. Elapsed time: 0.00 minutes. Runtime.totalMemory()=2027945984 INFO 16:13:38,458 HelpFormatter -

INFO 16:13:38,462 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.5-0-g36282e4, Compiled 2015/11/25 04:03:56 INFO 16:13:38,462 HelpFormatter - Copyright (c) 2010 The Broad Institute INFO 16:13:38,463 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk INFO 16:13:38,468 HelpFormatter - Program Args: -T HaplotypeCaller -R ./output/test//ref//merged_ref.fa -allowPotentiallyMisencodedQuals -I ./output/test//map/mapped.bam -o ./output/test//map/mapped.vcf.gz INFO 16:13:38,479 HelpFormatter - Executing as @.*** on Linux 2.6.32-696.30.1.el6.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_265-b11. INFO 16:13:38,480 HelpFormatter - Date/Time: 2021/03/12 16:13:38 INFO 16:13:38,480 HelpFormatter -

INFO 16:13:38,480 HelpFormatter -

INFO 16:13:38,595 GenomeAnalysisEngine - Strictness is SILENT INFO 16:13:39,470 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,470 HttpMethodDirector - Retrying request INFO 16:13:39,472 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,473 HttpMethodDirector - Retrying request INFO 16:13:39,474 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,475 HttpMethodDirector - Retrying request INFO 16:13:39,476 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,476 HttpMethodDirector - Retrying request INFO 16:13:39,478 HttpMethodDirector - I/O exception (java.net.ConnectException) caught when processing request: Network is unreachable (connect failed) INFO 16:13:39,478 HttpMethodDirector - Retrying request ERROR ------------------------------------------------------------------------------------------ ERROR A USER ERROR has occurred (version 3.5-0-g36282e4): ERROR ERROR This means that one or more arguments or inputs in your command are incorrect. ERROR The error message below tells you what is the problem. ERROR ERROR If the problem is an invalid argument, please check the online documentation guide ERROR (or rerun your command with --help) to view allowable command-line arguments for this tool. ERROR ERROR Visit our website and forum for extensive documentation and answers to ERROR commonly asked questions http://www.broadinstitute.org/gatk ERROR ERROR Please do NOT post this error to the GATK forum unless you have really tried to fix it yourself. ERROR ERROR MESSAGE: Fasta index file /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/./output/test/ref/merged_ref.fa.fai for reference /ldfssz1/ST_META/P18Z10200N0127_GRJ/liangweiting2/software/PStrain/test/./output/test/ref/merged_ref.fa does not exist. Please see http://gatkforums.broadinstitute.org/discussion/1601/how-can-i-prepare-a-fasta-file-to-use-as-reference for help creating it. ERROR

...................................... [E::hts_open_format] Failed to open file "./output/test//map/mapped.vcf.gz" : No such file or directory No need to merge in case of one sample.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wshuai294/PStrain/issues/4, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALS7DWXE6WHGBSJV4QY47JLTDHEWRANCNFSM4ZBZ3EVA .

weiting-liang commented 3 years ago

Thank you! The empty file "metaphlan2_output.txt" is in the ./output/test//metaphlan2 folder. I checked my run.sh and I have appointed metaphlan2's path: python ../install/scripts/PStrain.py -c config.txt -o ./output --dbdir ../install/db --prior ../install/db/prior_beta.pickle --bowtie2-build ../install/packages/bowtie2-2.3.1-legacy/bowtie2-build --bowtie2 ../install/packages/bowtie2-2.3.1-legacy/bowtie2 --samtools ../install/packages/SamTools-1.3.1/samtools --metaphlan2 ../install/packages/metaphlan2/metaphlan2.py --gatk ../install/packages/GATK_3.5/GenomeAnalysisTK.jar --picard ../install/packages/picard-tools-2.1.0/picard.jar It doesn’t seem to work. By the way, the python --version is 3.7.8.

Sincerely, Liang

wshuai294 commented 3 years ago

Hi Liang,

Sometimes software in the packages folder doesn't work, please kindly check if the software works. Maybe you need to download and install these third-party tools by yourself.

You can just run metaphlan2, if it works, the output should be:

SampleID Metaphlan2_Analysis

kBacteria 100.0 kBacteria|pProteobacteria 100.0 kBacteria|pProteobacteria|cGammaproteobacteria 100.0 kBacteria|pProteobacteria|cGammaproteobacteria|oEnterobacteriales 100.0 kBacteria|pProteobacteria|cGammaproteobacteria|oEnterobacteriales|fEnterobacteriaceae 100.0 kBacteria|pProteobacteria|cGammaproteobacteria|oEnterobacteriales|fEnterobacteriaceae|gEscherichia 100.0 kBacteria|pProteobacteria|cGammaproteobacteria|oEnterobacteriales|fEnterobacteriaceae|gEscherichia|sEscherichia_coli 100.0 kBacteria|pProteobacteria|cGammaproteobacteria|oEnterobacteriales|fEnterobacteriaceae|gEscherichia|s__Escherichia_coli|t__Escherichia_coli_unclassified 100.0

I hope it helps.

Best, WANG Shuai

weiting-liang commented 3 years ago

Hi Shuai !

Thank you so much! I get it! When I just ran metaphlan2, I found my test_2.fq broken. I updated the test_2.fq, and metaphlan2 work well. But the progress didn't work until I removed the old result file.

Thanks again for your kind help！ Liang