search

grenaud / glactools

command-line tools for the management of genotype likelihoods and allele counts
http://grenaud.github.io/glactools/
GNU General Public License v3.0
29 stars 2 forks source link

run error about Cannot write to /dev/stdout. #15

Closed shiyi-pan closed 4 years ago

shiyi-pan commented 4 years ago

I got a error when I run it. I hope you can do me a favour. I have two problem. first , when I run it in command line,my command is that: glactools vcf2acf --fai Gmax_275_v2.0.fa.fai --onlyGT Z-001.recode.vcf Z-001 > a.acf.gz but I got a warning like that : Warning: if several sites are rejected due to being empty, it could be due to the absence of the PL field, please rerun using the onlyGT flag. Program terminated gracefully. I have already used the onlyGT flag. I don't know why and here is my vcf file:

fileformat=VCFv4.2

CHROM POS ID REF ALT QUAL FILTER INFO FORMAT Z-001

1 4485 Chr01_4485 G A . . . GT 1/1 1 5564 Chr01_5564 C T . . . GT 0/0 1 5698 Chr01_5698 C T . . . GT 0/0 1 5899 Chr01_5899 T C . . . GT 1/1 the seconded problem. when I run it in a script , It don't have a result. my script is like that:

!/bin/bash

$ -cwd

$ -j y

$ -S /bin/bash

$ -l mem_free=4G

VCFTOOLS=/share/apps/vcftools/vcftools-0.1.15 GLACTOOLS=/ds3512/home/panyp/ruanjian/glactools/ SAMTOOLS=/ds3512/home/panyp/ruanjian/samtools-1.9/bin DIR=/ds3512/home/panyp/diff_genome/ $SAMTOOLS/samtools faidx $DIR/Gmax_275_v2.0.fa $VCFTOOLS --vcf chr01.recode.vcf --indv Z-001 --out Z-001 --recode $GLACTOOLS/glactools vcf2acf --fai $DIR/Gmax_275_v2.0.fa.fai --onlyGT Z-001.recode.vcf Z-001 > Z-001.acf.gz Z-001 is my sample ID. It generate a 'Z-001.acf.gz' file but the file can't be decompressed : gzip: Z-001.acf.gz: unexpected end of file. I check the log and find that: Filter used: Cutoffs : Minimum genotype quality (GQ) = 0 Minimum RMS of the mapping qualities (MQ) = 0 Minimum mapability = 0 Filter sites close to an indel = currently turned on/used Filter sites close labeled as repeat masked (RM) = currently turned on/used Filter sites close labeled as systematic error (SysErr) = currently turned on/used Minimum coverage = 0 Minimum coverage = 1000 Cannot write to /dev/stdout.

grenaud commented 4 years ago

Ok let's do this one problem at a time. The first, your command: glactools vcf2acf --fai Gmax_275_v2.0.fa.fai --onlyGT Z-001.recode.vcf Z-001 > a.acf.gz

Could you post the .fai file and the vcf file to a dropbox or something and please send me the link? I will have a look.

shiyi-pan commented 4 years ago

grenaud: Thank you for your help.I have upload my file in appendix. The file is big so that I upload a part of it. glactools_file.zip when I install the glactools,some warning happended like that: warning my GCC version is 8.3. and cmake is 3.5.2 . I also met a error like but when I type 'make' again, It when through whithout the error so I ignore it . I think maybe it's the reason why I'can get the right result. sed -e '/^static_libs=/s/@static_LIBS@/-lz -lm -lbz2 /ds3512/home/panyp/ruanjian/xz/lib/liblzma.so -lcurl/;s#@[^-][^@]*@##g' htslib.pc.in > htslib.pc.tmp sed: -e expression #1, char 47: unknown option to `s' make[1]: *** [htslib.pc.tmp] Error 1.

grenaud commented 4 years ago

Regarding the warnings, they are just warnings. I have on my todo list to take care of them.

I tried unzipping the file and I get: unzip glactools_file.zip Archive: glactools_file.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive.

could you post a bit of the vcf in zipped to a dropbox/link and the fasta index?

shiyi-pan commented 4 years ago

ok，I try to upload it again. chr01.zip

grenaud commented 4 years ago

Dear shiyi-pan, sorry for the late reply! I found a minor bug where it filtered out sites based on coverage even if there was no coverage. I have since fixed it. BTW, in the example you sent I noticed a discrepancy between the chromosome name used in the VCF file ('1') and the one in the fasta file ('Chr01'). You should use the exact fasta file used for mapping as to avoid any problems. Let me know if that works for you if so I will close this issue.

shiyi-pan commented 4 years ago

thank you,grenaud ，I downloaded the latest version of glactools and installed it, when I run the 'vcf2acf' in command line ,it still print like that:

Filter used: Cutoffs : Minimum genotype quality (GQ) = 0 Minimum RMS of the mapping qualities (MQ) = 0 Minimum mapability = 0 Filter sites close to an indel = currently turned on/used Filter sites close labeled as repeat masked (RM) = currently turned on/used Filter sites close labeled as systematic error (SysErr) = currently turned on/used Minimum coverage = 0 Minimum coverage = 1000

rejectIndel 0 rejectREFValidREF 0 rejectREFValidALT 0 rejectLOWCOV_REF 0 rejectMap20 0 rejectLOWMQ 0 rejectLOWQUAL 0 rejectCloseIndels 0 rejectSysERR 0 rejectRM 0 rejectREF_unknownGeno 0 rejectedEMPTY 0 Warning: if several sites are rejected due to being empty, it could be due to the absence of the PL field, please rerun using the onlyGT flag. Program terminated gracefully.

is that normal?

when I run it in script, it also didn't get a result. the log is here :

Filter used: Cutoffs : Minimum genotype quality (GQ) = 0 Minimum RMS of the mapping qualities (MQ) = 0 Minimum mapability = 0 Filter sites close to an indel = currently turned on/used Filter sites close labeled as repeat masked (RM) = currently turned on/used Filter sites close labeled as systematic error (SysErr) = currently turned on/used Minimum coverage = 0 Minimum coverage = 1000

Cannot write to /dev/stdout.

grenaud commented 4 years ago

I ran the program with the following command on your data: glactools vcf2acf --fai chr01/Chr01.fa.fai --onlyGT chr01/chr01.recode.vcf Z-001 > a.acf.gz

It returned a file: ` glactools view -h a.acf.gz|head

chr coord REF,ALT root anc Z-001

1 4485 G,A 0,0:0 0,0:0 0,2:0 1 5564 C,N 0,0:0 0,0:0 2,0:0 1 5698 C,N 0,0:0 0,0:0 2,0:0 1 5899 T,C 0,0:0 0,0:0 0,2:0 1 6035 G,A 0,0:0 0,0:0 0,2:0 1 6324 C,N 0,0:0 0,0:0 2,0:0 1 6333 C,N 0,0:0 0,0:0 2,0:0 1 6443 A,G 0,0:0 0,0:0 0,2:0 1 6474 C,N 0,0:0 0,0:0 2,0:0 `

which command did you use?

shiyi-pan commented 4 years ago

thank you for your attention,grenaud. I run the same command as yours and madifid the chromosome name in VCF file. I can get the same result like yours in command line,but I'm not sure it correct because the log end with that:

Program terminated gracefully

when I run the same command in my shell script, it told my that :

Filter used: Cutoffs : Minimum genotype quality (GQ) = 0 Minimum RMS of the mapping qualities (MQ) = 0 Minimum mapability = 0 Filter sites close to an indel = currently turned on/used Filter sites close labeled as repeat masked (RM) = currently turned on/used Filter sites close labeled as systematic error (SysErr) = currently turned on/used Minimum coverage = 0 Minimum coverage = 1000

Cannot write to /dev/stdout Error: GlacParser tried to read 4 bytes but got 0.

Here is my shell script:

!/bin/bash

#

$ -cwd

$ -j y

$ -S /bin/bash

#

$ -l mem_free=4G

#

export PATH=/ds3512/home/panyp/denovo/gcc-8.3/bin:$PATH export PATH=/ds3512/home/panyp/denovo/gcc-8.3/lib64:$PATH export LD_LIBRARY_PATH=/ds3512/home/panyp/denovo/gcc-8.3/lib64/:$LD_LIBRARY_PATH

export PATH=/ds3512/home/panyp/ruanjian/xz/bin:$PATH export LD_LIBRARY_PATH=/ds3512/home/panyp/ruanjian/xz/lib:$LD_LIBRARY_PATH export LDFLAGS="-L/ds3512/home/panyp/ruanjian/xz/lib" export CFLAGS="-I/ds3512/home/panyp/ruanjian/xz/include"

VCFTOOLS=/share/apps/vcftools/vcftools-0.1.15 GLACTOOLS=/ds3512/home/panyp/750cexu/balance_selection/glactools-master SAMTOOLS=/ds3512/home/panyp/ruanjian/samtools-1.9/bin PYTHON=/ds3512/home/panyp/ruanjian/python36/bin/python

DIR=/ds3512/home/panyp/diff_genome/

$SAMTOOLS/samtools faidx Chr01.fa

$VCFTOOLS --vcf chr01.recode.vcf --indv Z-001 --out Z-001 --recode

$PYTHON vcf_modify.py Z-001.recode.vcf > final-Z-001.recode.vcf $GLACTOOLS/glactools vcf2acf --fai Chr01.fa.fai --onlyGT final-Z-001.recode.vcf Z-001 > b.acf.gz $GLACTOOLS/glactools view -h b.acf.gz|tail

grenaud commented 4 years ago

Could you try to run the commands outside the shell script? Because with this, we do not know which command fails.

shiyi-pan commented 4 years ago

ok,here is my commands in command line and I can get the "a.acf.gz" file:

/ds3512/home/panyp/750cexu/balance_selection/glactools-master/glactools vcf2acf --fai Chr01.fa.fai --onlyGT final-Z-001.recode.vcf Z-001 > a.acf.gz

but I still got some log on my screen:

rejectIndel 0 rejectREFValidREF 0 rejectREFValidALT 0 rejectLOWCOV_REF 0 rejectMap20 0 rejectLOWMQ 0 rejectLOWQUAL 0 rejectCloseIndels 0 rejectSysERR 0 rejectRM 0 rejectREF_unknownGeno 0 rejectedEMPTY 0 Warning: if several sites are rejected due to being empty, it could be due to the absence of the PL field, please rerun using the onlyGT flag. Program terminated gracefully.

I'm not sure the command ended normally because there is a 'terminated' in it .

grenaud commented 4 years ago

It seems that the command ran ok, no sites were filtered out it seems.

shiyi-pan commented 4 years ago

Thank you for you reply,but when I run the same code in shell srcipt ,it report an error,here is my script:

!/bin/bash

#

$ -cwd

$ -j y

$ -S /bin/bash

#

$ -l mem_free=4G

#

export PATH=/ds3512/home/panyp/denovo/gcc-8.3/bin:$PATH export PATH=/ds3512/home/panyp/denovo/gcc-8.3/lib64:$PATH export LD_LIBRARY_PATH=/ds3512/home/panyp/denovo/gcc-8.3/lib64/:$LD_LIBRARY_PATH

export PATH=/ds3512/home/panyp/ruanjian/xz/bin:$PATH export LD_LIBRARY_PATH=/ds3512/home/panyp/ruanjian/xz/lib:$LD_LIBRARY_PATH export LDFLAGS="-L/ds3512/home/panyp/ruanjian/xz/lib" export CFLAGS="-I/ds3512/home/panyp/ruanjian/xz/include"

VCFTOOLS=/share/apps/vcftools/vcftools-0.1.15 GLACTOOLS=/ds3512/home/panyp/750cexu/balance_selection/glactools-master SAMTOOLS=/ds3512/home/panyp/ruanjian/samtools-1.9/bin PYTHON=/ds3512/home/panyp/ruanjian/python36/bin/python

DIR=/ds3512/home/panyp/diff_genome/

$SAMTOOLS/samtools faidx Chr01.fa

$VCFTOOLS --vcf chr01.recode.vcf --indv Z-001 --out Z-001 --recode

$PYTHON vcf_modify.py Z-001.recode.vcf > final-Z-001.recode.vcf $GLACTOOLS/glactools vcf2acf --fai Chr01.fa.fai --onlyGT final-Z-001.recode.vcf Z-001 > b.acf.gz $GLACTOOLS/glactools view -h b.acf.gz|tail

here is my error:

Filter used: Cutoffs : Minimum genotype quality (GQ) = 0 Minimum RMS of the mapping qualities (MQ) = 0 Minimum mapability = 0 Filter sites close to an indel = currently turned on/used Filter sites close labeled as repeat masked (RM) = currently turned on/used Filter sites close labeled as systematic error (SysErr) = currently turned on/used Minimum coverage = 0 Minimum coverage = 1000

Cannot write to /dev/stdout Error: GlacParser tried to read 4 bytes but got 0

grenaud commented 4 years ago

Let's try viewing the file, can you run:

/ds3512/home/panyp/750cexu/balance_selection/glactools-master/glactools view a.acf.gz | head

and post the output?

shiyi-pan commented 4 years ago

ok, thank you,grenaud. there are two files "a.acf.gz" and "b.acf.gz" . the a.acf.gz is the result I got in command line and b.acf.gz is the result I got in shell script. here is their state . because the b.acf.gz is empty , I can't upload it.

[panyp@NCSI balance_selection]$ ll a.acf.gz b.acf.gz -rw-rw-r-- 1 panyp panyp 541420 Jul 27 09:39 a.acf.gz -rw-rw-r-- 1 panyp panyp 0 Jul 27 22:07 b.acf.gz

a.acf.gz

here is the result I run the "view" command:

[panyp@NCSI balance_selection]$ /ds3512/home/panyp/750cexu/balance_selection/glactools-master/glactools view a.acf.gz | head Chr01 4485 G,A 0,0:0 0,0:0 0,2:0 Chr01 5564 C,N 0,0:0 0,0:0 2,0:0 Chr01 5698 C,N 0,0:0 0,0:0 2,0:0 Chr01 5899 T,C 0,0:0 0,0:0 0,2:0 Chr01 6035 G,A 0,0:0 0,0:0 0,2:0 Chr01 6324 C,N 0,0:0 0,0:0 2,0:0 Chr01 6333 C,N 0,0:0 0,0:0 2,0:0 Chr01 6443 A,G 0,0:0 0,0:0 0,2:0 Chr01 6474 C,N 0,0:0 0,0:0 2,0:0 Chr01 6483 G,N 0,0:0 0,0:0 2,0:0

[panyp@NCSI balance_selection]$ /ds3512/home/panyp/750cexu/balance_selection/glactools-master/glactools view b.acf.gz | head Error: GlacParser tried to read 4 bytes but got 0.

grenaud commented 4 years ago

So it seems that the program works. I do not know what happens with the script, there could be an environment issue. I would need to login to diagnose. However, as nothing is wrong with the program, I would like to close the issue if that is ok with you?

shiyi-pan commented 4 years ago

ok,thank you for your help. I will Contact my server administrator.