search

Illumina / hap.py

Haplotype VCF comparison tools
Other
420 stars 124 forks source link

CalledProcessError: Command xcmp #32

Open simoncchu opened 6 years ago

simoncchu commented 6 years ago

Hi, I am comparing two vcf file, and get the following errors:

[W] overlapping records at 1:6799149 for sample 0
[W] Variants that overlap on the reference allele: 1346
[I] Total VCF records:         4029532
[I] Non-reference VCF records: 4029532
[W] overlapping records at 1:8149316 for sample 0
[W] Variants that overlap on the reference allele: 464
[I] Total VCF records:         4003155
[I] Non-reference VCF records: 4003155
2018-01-10 12:31:59,960 WARNING  No calls for location MT in query!
2018-01-10 12:34:22,279 WARNING  [W] too many AD fields at 1:49573365 max_ad = 6 retrieved: 7
2018-01-10 12:34:22,279 WARNING  *** Error in `xcmp': free(): invalid next size (fast): 0x0000000002c96e90 ***
2018-01-10 12:34:22,280 WARNING  ======= Backtrace: =========
2018-01-10 12:34:22,280 WARNING  /usr/lib64/libc.so.6(+0x7d023)[0x7efdc5c05023]
2018-01-10 12:34:22,280 WARNING  xcmp(_ZN7variant13VariantReader7advanceEv+0x211f)[0x56eac3]
2018-01-10 12:34:22,280 WARNING  xcmp(main+0x2222)[0x533839]
2018-01-10 12:34:22,280 WARNING  /usr/lib64/libc.so.6(__libc_start_main+0xf5)[0x7efdc5ba9b15]
2018-01-10 12:34:22,280 WARNING  xcmp[0x52fd41]
2018-01-10 12:34:22,280 WARNING  ======= Memory map: ========
2018-01-10 12:34:22,280 WARNING  00400000-007a0000 r-xp 00000000 00:25 20026347337                        tools/hap.py-build/bin/xcmp
...
...
2018-01-10 12:47:17,996 WARNING  ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
2018-01-10 12:47:17,996 ERROR    Exception when running <function xcmpWrapper at 0x7fa633357668>:
2018-01-10 12:47:17,996 ERROR    ------------------------------------------------------------
2018-01-10 12:47:17,997 ERROR    Traceback (most recent call last):
2018-01-10 12:47:17,997 ERROR      File "tools/hap.py-build/lib/python27/Tools/parallel.py", line 71, in parMapper
2018-01-10 12:47:17,997 ERROR        return arg[1]['fun'](arg[0], *arg[1]['args'], **arg[1]['kwargs'])
2018-01-10 12:47:17,997 ERROR      File "tools/hap.py-build/lib/python27/Haplo/xcmp.py", line 69, in xcmpWrapper
2018-01-10 12:47:17,997 ERROR        subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe)
2018-01-10 12:47:17,997 ERROR      File "/home/cc439/anaconda2/envs/hornet/lib/python2.7/subprocess.py", line 186, in check_call
2018-01-10 12:47:17,997 ERROR        raise CalledProcessError(retcode, cmd)
2018-01-10 12:47:17,997 ERROR    CalledProcessError: Command 'xcmp /tmp/truth.ppooeVs5.vcf.gz /tmp/query.ppR_g6QK.vcf.gz -l 3:46861889-55529583 -o /tmp/result.3:46861889-55529583yNJudr.bcf -r refdata-b37-2.1.0/fasta/genome.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status -6
2018-01-10 12:47:17,997 ERROR    ------------------------------------------------------------

Any possible reasons for this?

pkrusche commented 6 years ago

There seems to be an issue with the input VCF file where the number of AD fields is not what xcmp expects.

I think two things can be done -- some of this is on our backlog for the next release.

Hope this helps!

If you would like me to look into this further, could you perhaps share a minimal VCF that reproduces the error? Thanks!

polinanvkv commented 5 years ago

I get kinda the same issue:

[W] overlapping records at chr1:145311562 for sample 0
[W] Variants that overlap on the reference allele: 8
[I] Total VCF records:         416689
[I] Non-reference VCF records: 416689
[W] variant at chr1:931131 has more than one base of reference padding 
[W] overlapping records at chr1:77558661 for sample 0
[W] Variants that have >1 base of reference padding: 5202
[W] Variants that overlap on the reference allele: 44
[I] Total VCF records:         249223
[I] Non-reference VCF records: 249223
2019-08-14 11:43:37,962 WARNING  Cannot query reference sequence at :114435806-114435806
2019-08-14 11:43:37,963 ERROR    Exception when running <function xcmpWrapper at 0x7f992b890b90>:
2019-08-14 11:43:37,963 ERROR    ------------------------------------------------------------
2019-08-14 11:43:37,964 ERROR    Traceback (most recent call last):
2019-08-14 11:43:37,964 ERROR      File "/home/polina/hap.py-install/lib/python27/Tools/parallel.py", line 72, in parMapper
2019-08-14 11:43:37,964 ERROR        return arg[1]['fun'](arg[0], *arg[1]['args'], **arg[1]['kwargs'])
2019-08-14 11:43:37,964 ERROR      File "/home/polina/hap.py-install/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper
2019-08-14 11:43:37,965 ERROR        subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe)
2019-08-14 11:43:37,965 ERROR      File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
2019-08-14 11:43:37,965 ERROR        raise CalledProcessError(retcode, cmd)
2019-08-14 11:43:37,965 ERROR    CalledProcessError: Command 'xcmp /tmp/truth.ppVprim2.vcf.gz /tmp/query.ppyJTgaX.vcf.gz -l chr13:102911813-2147483647 -o /tmp/result.chr13:102911813-21474836472oM7PT.bcf -r ref/hg38.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1
2019-08-14 11:43:37,965 ERROR    ------------------------------------------------------------
...
2019-08-14 11:43:56,982 ERROR    ------------------------------------------------------------
2019-08-14 11:43:57,503 WARNING  Cannot query reference sequence at :59003694-59003694
2019-08-14 11:43:57,503 ERROR    Exception when running <function xcmpWrapper at 0x7f992b890b90>:
2019-08-14 11:43:57,503 ERROR    ------------------------------------------------------------
2019-08-14 11:43:57,503 ERROR    Traceback (most recent call last):
2019-08-14 11:43:57,503 ERROR      File "/home/polina/hap.py-install/lib/python27/Tools/parallel.py", line 72, in parMapper
2019-08-14 11:43:57,503 ERROR        return arg[1]['fun'](arg[0], *arg[1]['args'], **arg[1]['kwargs'])
2019-08-14 11:43:57,503 ERROR      File "/home/polina/hap.py-install/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper
2019-08-14 11:43:57,504 ERROR        subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe)
2019-08-14 11:43:57,504 ERROR      File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
2019-08-14 11:43:57,504 ERROR        raise CalledProcessError(retcode, cmd)
2019-08-14 11:43:57,504 ERROR    CalledProcessError: Command 'xcmp /tmp/truth.ppVprim2.vcf.gz /tmp/query.ppyJTgaX.vcf.gz -l chrY:59003031-2147483647 -o /tmp/result.chrY:59003031-2147483647gCLH20.bcf -r ref/hg38.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1
2019-08-14 11:43:57,504 ERROR    ------------------------------------------------------------
2019-08-14 11:43:57,512 ERROR    One of the xcmp jobs failed.
2019-08-14 11:43:57,512 ERROR    Traceback (most recent call last):
2019-08-14 11:43:57,512 ERROR      File "/home/polina/hap.py-install/bin/hap.py", line 511, in <module>
2019-08-14 11:43:57,512 ERROR        main()
2019-08-14 11:43:57,512 ERROR      File "/home/polina/hap.py-install/bin/hap.py", line 434, in main
2019-08-14 11:43:57,512 ERROR        raise Exception("One of the xcmp jobs failed.")
2019-08-14 11:43:57,512 ERROR    Exception: One of the xcmp jobs failed.

What could be the problem?

Lenbok commented 5 years ago

Have you tried using vcfeval instead of xcmp as the comparison engine?

24natasya commented 5 years ago

Hi ! I seems to have the same error by using octopus vcfs file. i figured that it can only be used by using octopus.legacy.vcf. Is there any way that this problem can be sorted out without having the need to use the legacy file ? I already tried --no-leftshift no-decompose --gender=none & --no-leftshift no--decompose --engine vcfeval

019-08-19 07:48:24,895 ERROR Preprocess command preprocess /tmp/tmpaE3POe.vcf.gz: -l 3:1-4303967 -o /tmp/input.3:1-43039676GkMrn.prep.vcf.gz -V 1 -L 1 -r /export/Projects/2019_MLVarCaller/01_BAMs/hs37d5.fa failed. Outputs are here /tmp/stdoutQUfSVp.log / /tmp/stderrBfEqNT.log 2019-08-19 07:48:24,910 ERROR Filter 'q10,LBQ' is not in the VCF header. This will break VCF writing, at 3:3170077 2019-08-19 07:48:24,910 ERROR Exception when running <function preprocessWrapper at 0x7f1d359690c8>: 2019-08-19 07:48:24,910 ERROR ------------------------------------------------------------ 2019-08-19 07:48:24,911 ERROR Traceback (most recent call last): 2019-08-19 07:48:24,911 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-19 07:48:24,911 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], *arg[1]['kwargs']) 2019-08-19 07:48:24,911 ERROR File "/opt/hap.py/lib/python27/Haplo/partialcredit.py", line 67, in preprocessWrapper 2019-08-19 07:48:24,911 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-19 07:48:24,912 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-19 07:48:24,912 ERROR raise CalledProcessError(retcode, cmd) 2019-08-19 07:48:24,912 ERROR CalledProcessError: Command 'preprocess /tmp/tmpaE3POe.vcf.gz: -l 3:1-4303967 -o /tmp/input.3:1-43039676GkMrn.prep.vcf.gz -V 1 -L 1 -r /export/Projects/2019_MLVarCaller/01_BAMs/hs37d5.fa' returned non-zero exit status 1 2019-08-19 07:48:24,912 ERROR ------------------------------

DaraEleb commented 5 years ago

Hi, I'm having the same issue with both vcfeval and xcmp.

WITH xcmp:

docker run -v "${INPUT_DIR}":"/input" -v "${OUTPUT_DIR}:/output" pkrusche/hap.py /opt/hap.py/bin/hap.py /input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2_highconf.vcf.gz /output/output.vcf.gz -f "/input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2.1_highconf.bed" -r "/input/Homo_sapiens.GRCh38.dna.primary_assembly.fa" -o "/output/happy.output"
2019-08-29 19:36:46,581 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. 2019-08-29 19:36:46,589 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. [W] overlapping records at 1:30934817 for sample 0 [W] Variants that overlap on the reference allele: 328 [I] Total VCF records: 4154255 [I] Non-reference VCF records: 4154255 [I] Total VCF records: 684797 [I] Non-reference VCF records: 341918 2019-08-29 19:41:08,943 WARNING Cannot query reference sequence at :114426602-114426602 2019-08-29 19:41:08,943 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:08,943 ERROR ------------------------------------------------------------ 2019-08-29 19:41:08,944 ERROR Traceback (most recent call last): 2019-08-29 19:41:08,944 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:08,944 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:08,944 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:08,944 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:08,944 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:08,945 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:08,945 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 13:112782711-2147483647 -o /tmp/result.13:112782711-21474836475RpfIT.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:08,945 ERROR ------------------------------------------------------------ 2019-08-29 19:41:08,972 WARNING Cannot query reference sequence at :133797582-133797582 2019-08-29 19:41:08,972 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:08,972 ERROR ------------------------------------------------------------ 2019-08-29 19:41:08,972 ERROR Traceback (most recent call last): 2019-08-29 19:41:08,972 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:08,973 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:08,973 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:08,973 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:08,973 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:08,973 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:08,974 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 10:133547689-2147483647 -o /tmp/result.10:133547689-214748364718i9GX.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:08,974 ERROR ------------------------------------------------------------ 2019-08-29 19:41:12,581 WARNING Cannot query reference sequence at :133277982-133277982 2019-08-29 19:41:12,582 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:12,582 ERROR ------------------------------------------------------------ 2019-08-29 19:41:12,582 ERROR Traceback (most recent call last): 2019-08-29 19:41:12,582 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:12,582 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:12,582 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:12,582 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:12,582 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:12,583 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:12,583 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 12:132056501-2147483647 -o /tmp/result.12:132056501-2147483647Ss0uYm.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:12,583 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,738 WARNING Cannot query reference sequence at :107043733-107043733 2019-08-29 19:41:13,739 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:13,739 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,739 ERROR Traceback (most recent call last): 2019-08-29 19:41:13,739 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:13,740 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:13,740 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:13,740 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:13,741 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:13,742 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:13,742 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 14:105460025-107069422 -o /tmp/result.14:105460025-107069422otRyoB.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:13,742 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,772 WARNING Cannot query reference sequence at :249058568-249058568 2019-08-29 19:41:13,772 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:13,772 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,772 ERROR Traceback (most recent call last): 2019-08-29 19:41:13,773 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:13,773 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:13,773 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:13,773 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:13,773 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:13,774 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:13,774 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 1:246383959-2147483647 -o /tmp/result.1:246383959-2147483647R46qGI.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:13,774 ERROR ------------------------------------------------------------ 2019-08-29 19:41:15,032 WARNING Cannot query reference sequence at :107069440-107069440 2019-08-29 19:41:15,032 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:15,032 ERROR ------------------------------------------------------------ 2019-08-29 19:41:15,032 ERROR Traceback (most recent call last): 2019-08-29 19:41:15,032 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:15,036 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:15,036 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:15,036 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:15,036 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:15,036 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:15,037 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 14:107069423-2147483647 -o /tmp/result.14:107069423-2147483647ro1_Iq.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:15,037 ERROR ------------------------------------------------------------ 2019-08-29 19:41:18,859 WARNING Cannot query reference sequence at :58618128-58618135 2019-08-29 19:41:18,860 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:18,860 ERROR ------------------------------------------------------------ 2019-08-29 19:41:18,860 ERROR Traceback (most recent call last): 2019-08-29 19:41:18,860 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:18,860 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:18,860 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:18,860 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:18,860 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:18,860 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:18,861 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 19:57195434-58623085 -o /tmp/result.19:57195434-58623085IKZkws.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:18,861 ERROR ------------------------------------------------------------ 2019-08-29 19:41:20,135 WARNING Cannot query reference sequence at :58623248-58623248 2019-08-29 19:41:20,135 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:20,135 ERROR ------------------------------------------------------------ 2019-08-29 19:41:20,135 ERROR Traceback (most recent call last): 2019-08-29 19:41:20,135 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:20,135 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:20,135 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:20,135 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:20,136 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:20,136 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:20,136 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 19:58623086-2147483647 -o /tmp/result.19:58623086-2147483647wYfZWT.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:20,136 ERROR ------------------------------------------------------------ 2019-08-29 19:41:21,451 WARNING Cannot query reference sequence at :101993020-101993020 2019-08-29 19:41:21,451 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:21,451 ERROR ------------------------------------------------------------ 2019-08-29 19:41:21,452 ERROR Traceback (most recent call last): 2019-08-29 19:41:21,452 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:21,452 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:21,452 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:21,452 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:21,452 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:21,453 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:21,453 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 15:101333106-2147483647 -o /tmp/result.15:101333106-21474836473Gv1C2.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:21,453 ERROR ------------------------------------------------------------ 2019-08-29 19:41:29,211 WARNING Cannot query reference sequence at :242194149-242194149 2019-08-29 19:41:29,212 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:29,212 ERROR ------------------------------------------------------------ 2019-08-29 19:41:29,212 ERROR Traceback (most recent call last): 2019-08-29 19:41:29,212 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:29,212 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:29,212 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:29,212 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:29,212 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:29,213 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:29,213 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 2:239272763-2147483647 -o /tmp/result.2:239272763-2147483647GZPbe0.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:29,213 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,638 WARNING Cannot query reference sequence at :46710276-46710276 2019-08-29 19:41:30,638 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:30,638 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,638 ERROR Traceback (most recent call last): 2019-08-29 19:41:30,638 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:30,639 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:30,639 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:30,639 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:30,639 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:30,639 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:30,640 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 21:46085921-46770254 -o /tmp/result.21:46085921-467702545OHULl.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:30,640 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,841 WARNING Cannot query reference sequence at :50818714-50818714 2019-08-29 19:41:30,842 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:30,842 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,842 ERROR Traceback (most recent call last): 2019-08-29 19:41:30,842 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:30,842 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:30,843 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:30,843 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:30,843 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:30,843 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:30,843 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 22:50704529-2147483647 -o /tmp/result.22:50704529-2147483647Yzbye1.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:30,844 ERROR ------------------------------------------------------------ 2019-08-29 19:41:31,895 WARNING Cannot query reference sequence at :46770531-46770531 2019-08-29 19:41:31,895 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:31,895 ERROR ------------------------------------------------------------ 2019-08-29 19:41:31,896 ERROR Traceback (most recent call last): 2019-08-29 19:41:31,896 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:31,896 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:31,896 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:31,896 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:31,896 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:31,896 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:31,896 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 21:46770255-47706397 -o /tmp/result.21:46770255-47706397dN9l1A.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:31,896 ERROR ------------------------------------------------------------ 2019-08-29 19:41:33,260 WARNING Cannot query reference sequence at :47708253-47708253 2019-08-29 19:41:33,260 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:33,260 ERROR ------------------------------------------------------------ 2019-08-29 19:41:33,260 ERROR Traceback (most recent call last): 2019-08-29 19:41:33,260 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:33,260 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:33,260 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:33,261 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:33,261 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:33,261 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:33,261 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 21:47706398-2147483647 -o /tmp/result.21:47706398-2147483647dvwcna.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:33,261 ERROR ------------------------------------------------------------ 2019-08-29 19:41:36,917 WARNING Cannot query reference sequence at :170806447-170806447 2019-08-29 19:41:36,918 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:36,918 ERROR ------------------------------------------------------------ 2019-08-29 19:41:36,918 ERROR Traceback (most recent call last): 2019-08-29 19:41:36,918 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:36,919 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:36,919 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:36,919 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:36,919 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:36,919 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:36,920 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 6:167912504-2147483647 -o /tmp/result.6:167912504-2147483647Z6NLeI.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:36,920 ERROR ------------------------------------------------------------ 2019-08-29 19:41:37,303 WARNING Cannot query reference sequence at :190214677-190214677 2019-08-29 19:41:37,303 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:37,304 ERROR ------------------------------------------------------------ 2019-08-29 19:41:37,304 ERROR Traceback (most recent call last): 2019-08-29 19:41:37,304 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:37,305 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:37,305 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:37,305 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:37,305 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:37,305 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:37,306 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 4:188114755-2147483647 -o /tmp/result.4:188114755-2147483647Pb56lZ.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:37,306 ERROR ------------------------------------------------------------ 2019-08-29 19:41:38,527 WARNING Cannot query reference sequence at :138401849-138401849 2019-08-29 19:41:38,527 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:38,527 ERROR ------------------------------------------------------------ 2019-08-29 19:41:38,527 ERROR Traceback (most recent call last): 2019-08-29 19:41:38,528 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:38,528 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:38,528 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:38,528 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:38,528 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:38,529 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:38,529 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 9:136762355-139076411 -o /tmp/result.9:136762355-13907641113yu5u.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:38,529 ERROR ------------------------------------------------------------ 2019-08-29 19:41:39,733 WARNING Cannot query reference sequence at :139079107-139079107 2019-08-29 19:41:39,733 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:39,733 ERROR ------------------------------------------------------------ 2019-08-29 19:41:39,733 ERROR Traceback (most recent call last): 2019-08-29 19:41:39,733 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:39,733 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:39,733 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:39,734 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:39,734 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:39,734 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:39,734 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 9:139076412-2147483647 -o /tmp/result.9:139076412-2147483647zYY_Q2.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:39,734 ERROR ------------------------------------------------------------ 2019-08-29 19:41:41,171 WARNING Cannot query reference sequence at :145147006-145147006 2019-08-29 19:41:41,171 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:41,171 ERROR ------------------------------------------------------------ 2019-08-29 19:41:41,171 ERROR Traceback (most recent call last): 2019-08-29 19:41:41,171 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:41,172 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], **arg[1]['kwargs']) 2019-08-29 19:41:41,172 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:41,172 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:41,172 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:41,172 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:41,172 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 8:143333332-2147483647 -o /tmp/result.8:143333332-2147483647MSsCPu.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:41,172 ERROR ------------------------------------------------------------ 2019-08-29 19:41:45,982 ERROR One of the xcmp jobs failed. 2019-08-29 19:41:45,982 ERROR Traceback (most recent call last): 2019-08-29 19:41:45,982 ERROR File "/opt/hap.py/bin/hap.py", line 508, in 2019-08-29 19:41:45,983 ERROR main() 2019-08-29 19:41:45,983 ERROR File "/opt/hap.py/bin/hap.py", line 431, in main 2019-08-29 19:41:45,983 ERROR raise Exception("One of the xcmp jobs failed.") 2019-08-29 19:41:45,983 ERROR Exception: One of the xcmp jobs failed.

With vcfeval: docker run -v "${INPUT_DIR}":"/input" -v "${OUTPUT_DIR}:/output" pkrusche/hap.py /opt/hap.py/bin/hap.py /input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2_highconf.vcf.gz /output/output.vcf.gz -f "/input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2.1_highconf.bed" -r "/input/Homo_sapiens.GRCh38.dna.primary_assembly.fa" -o "/output/happy.output" --no-leftshift --no-decompose --engine=vcfeval --engine-vcfeval-template "/input/GRCh38_hs38d1.sdf" 2019-08-29 19:46:59,817 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. 2019-08-29 19:46:59,824 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. [W] overlapping records at 1:30934817 for sample 0 [W] Variants that overlap on the reference allele: 328 [I] Total VCF records: 4154255 [I] Non-reference VCF records: 4154255 [I] Total VCF records: 684797 [I] Non-reference VCF records: 341918 2019-08-29 19:49:51,018 ERROR Error running rtg tools / vcfeval. Return code was 1, output: / Error: There were no sequence names in common between the reference and the supplied variant sets. Check the correct reference is being used.

2019-08-29 19:49:51,019 ERROR Traceback (most recent call last): 2019-08-29 19:49:51,019 ERROR File "/opt/hap.py/bin/hap.py", line 508, in 2019-08-29 19:49:51,019 ERROR main() 2019-08-29 19:49:51,019 ERROR File "/opt/hap.py/bin/hap.py", line 452, in main 2019-08-29 19:49:51,019 ERROR tempfiles += Haplo.vcfeval.runVCFEval(args.vcf1, args.vcf2, output_name, args) 2019-08-29 19:49:51,019 ERROR File "/opt/hap.py/lib/python27/Haplo/vcfeval.py", line 129, in runVCFEval 2019-08-29 19:49:51,019 ERROR raise Exception("Error running rtg tools / vcfeval. Return code was %i, output: %s / %s \n" % (rc, o, e)) 2019-08-29 19:49:51,019 ERROR Exception: Error running rtg tools / vcfeval. Return code was 1, output: / Error: There were no sequence names in common between the reference and the supplied variant sets. Check the correct reference is being used.

It says there were no sequence names in common between reference and supplied variant sets but I used the same reference for variant calling.

docker run -v "${INPUT_DIR}":"/input" -v "${OUTPUT_DIR}:/output" gcr.io/deepvariant-docker/deepvariant:"${BIN_VERSION}" /opt/deepvariant/bin/run_deepvariant --model_type=WES --ref=/input/Homo_sapiens.GRCh38.dna.primary_assembly.fa --reads=/input/SRR2635048_1.RG.sorted.bam --output_vcf=/output/output.vcf.gz --output_gvcf=/output/output.g.vcf.gz --num_shards=30

Is it something I am doing wrong?

24natasya commented 5 years ago

Hi, I'm having the same issue with both vcfeval and xcmp.

WITH xcmp:

docker run -v "${INPUT_DIR}":"/input" -v "${OUTPUT_DIR}:/output" pkrusche/hap.py /opt/hap.py/bin/hap.py /input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2_highconf.vcf.gz /output/output.vcf.gz -f "/input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2.1_highconf.bed" -r "/input/Homo_sapiens.GRCh38.dna.primary_assembly.fa" -o "/output/happy.output" 2019-08-29 19:36:46,581 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. 2019-08-29 19:36:46,589 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. [W] overlapping records at 1:30934817 for sample 0 [W] Variants that overlap on the reference allele: 328 [I] Total VCF records: 4154255 [I] Non-reference VCF records: 4154255 [I] Total VCF records: 684797 [I] Non-reference VCF records: 341918 2019-08-29 19:41:08,943 WARNING Cannot query reference sequence at :114426602-114426602 2019-08-29 19:41:08,943 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:08,943 ERROR ------------------------------------------------------------ 2019-08-29 19:41:08,944 ERROR Traceback (most recent call last): 2019-08-29 19:41:08,944 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:08,944 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:08,944 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:08,944 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:08,944 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:08,945 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:08,945 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 13:112782711-2147483647 -o /tmp/result.13:112782711-21474836475RpfIT.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:08,945 ERROR ------------------------------------------------------------ 2019-08-29 19:41:08,972 WARNING Cannot query reference sequence at :133797582-133797582 2019-08-29 19:41:08,972 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:08,972 ERROR ------------------------------------------------------------ 2019-08-29 19:41:08,972 ERROR Traceback (most recent call last): 2019-08-29 19:41:08,972 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:08,973 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:08,973 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:08,973 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:08,973 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:08,973 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:08,974 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 10:133547689-2147483647 -o /tmp/result.10:133547689-214748364718i9GX.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:08,974 ERROR ------------------------------------------------------------ 2019-08-29 19:41:12,581 WARNING Cannot query reference sequence at :133277982-133277982 2019-08-29 19:41:12,582 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:12,582 ERROR ------------------------------------------------------------ 2019-08-29 19:41:12,582 ERROR Traceback (most recent call last): 2019-08-29 19:41:12,582 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:12,582 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:12,582 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:12,582 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:12,582 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:12,583 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:12,583 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 12:132056501-2147483647 -o /tmp/result.12:132056501-2147483647Ss0uYm.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:12,583 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,738 WARNING Cannot query reference sequence at :107043733-107043733 2019-08-29 19:41:13,739 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:13,739 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,739 ERROR Traceback (most recent call last): 2019-08-29 19:41:13,739 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:13,740 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:13,740 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:13,740 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:13,741 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:13,742 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:13,742 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 14:105460025-107069422 -o /tmp/result.14:105460025-107069422otRyoB.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:13,742 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,772 WARNING Cannot query reference sequence at :249058568-249058568 2019-08-29 19:41:13,772 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:13,772 ERROR ------------------------------------------------------------ 2019-08-29 19:41:13,772 ERROR Traceback (most recent call last): 2019-08-29 19:41:13,773 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:13,773 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:13,773 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:13,773 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:13,773 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:13,774 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:13,774 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 1:246383959-2147483647 -o /tmp/result.1:246383959-2147483647R46qGI.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:13,774 ERROR ------------------------------------------------------------ 2019-08-29 19:41:15,032 WARNING Cannot query reference sequence at :107069440-107069440 2019-08-29 19:41:15,032 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:15,032 ERROR ------------------------------------------------------------ 2019-08-29 19:41:15,032 ERROR Traceback (most recent call last): 2019-08-29 19:41:15,032 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:15,036 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:15,036 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:15,036 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:15,036 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:15,036 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:15,037 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 14:107069423-2147483647 -o /tmp/result.14:107069423-2147483647ro1_Iq.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:15,037 ERROR ------------------------------------------------------------ 2019-08-29 19:41:18,859 WARNING Cannot query reference sequence at :58618128-58618135 2019-08-29 19:41:18,860 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:18,860 ERROR ------------------------------------------------------------ 2019-08-29 19:41:18,860 ERROR Traceback (most recent call last): 2019-08-29 19:41:18,860 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:18,860 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:18,860 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:18,860 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:18,860 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:18,860 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:18,861 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 19:57195434-58623085 -o /tmp/result.19:57195434-58623085IKZkws.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:18,861 ERROR ------------------------------------------------------------ 2019-08-29 19:41:20,135 WARNING Cannot query reference sequence at :58623248-58623248 2019-08-29 19:41:20,135 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:20,135 ERROR ------------------------------------------------------------ 2019-08-29 19:41:20,135 ERROR Traceback (most recent call last): 2019-08-29 19:41:20,135 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:20,135 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:20,135 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:20,135 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:20,136 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:20,136 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:20,136 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 19:58623086-2147483647 -o /tmp/result.19:58623086-2147483647wYfZWT.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:20,136 ERROR ------------------------------------------------------------ 2019-08-29 19:41:21,451 WARNING Cannot query reference sequence at :101993020-101993020 2019-08-29 19:41:21,451 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:21,451 ERROR ------------------------------------------------------------ 2019-08-29 19:41:21,452 ERROR Traceback (most recent call last): 2019-08-29 19:41:21,452 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:21,452 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:21,452 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:21,452 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:21,452 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:21,453 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:21,453 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 15:101333106-2147483647 -o /tmp/result.15:101333106-21474836473Gv1C2.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:21,453 ERROR ------------------------------------------------------------ 2019-08-29 19:41:29,211 WARNING Cannot query reference sequence at :242194149-242194149 2019-08-29 19:41:29,212 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:29,212 ERROR ------------------------------------------------------------ 2019-08-29 19:41:29,212 ERROR Traceback (most recent call last): 2019-08-29 19:41:29,212 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:29,212 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:29,212 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:29,212 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:29,212 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:29,213 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:29,213 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 2:239272763-2147483647 -o /tmp/result.2:239272763-2147483647GZPbe0.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:29,213 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,638 WARNING Cannot query reference sequence at :46710276-46710276 2019-08-29 19:41:30,638 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:30,638 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,638 ERROR Traceback (most recent call last): 2019-08-29 19:41:30,638 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:30,639 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:30,639 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:30,639 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:30,639 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:30,639 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:30,640 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 21:46085921-46770254 -o /tmp/result.21:46085921-467702545OHULl.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:30,640 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,841 WARNING Cannot query reference sequence at :50818714-50818714 2019-08-29 19:41:30,842 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:30,842 ERROR ------------------------------------------------------------ 2019-08-29 19:41:30,842 ERROR Traceback (most recent call last): 2019-08-29 19:41:30,842 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:30,842 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:30,843 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:30,843 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:30,843 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:30,843 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:30,843 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 22:50704529-2147483647 -o /tmp/result.22:50704529-2147483647Yzbye1.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:30,844 ERROR ------------------------------------------------------------ 2019-08-29 19:41:31,895 WARNING Cannot query reference sequence at :46770531-46770531 2019-08-29 19:41:31,895 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:31,895 ERROR ------------------------------------------------------------ 2019-08-29 19:41:31,896 ERROR Traceback (most recent call last): 2019-08-29 19:41:31,896 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:31,896 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:31,896 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:31,896 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:31,896 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:31,896 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:31,896 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 21:46770255-47706397 -o /tmp/result.21:46770255-47706397dN9l1A.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:31,896 ERROR ------------------------------------------------------------ 2019-08-29 19:41:33,260 WARNING Cannot query reference sequence at :47708253-47708253 2019-08-29 19:41:33,260 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:33,260 ERROR ------------------------------------------------------------ 2019-08-29 19:41:33,260 ERROR Traceback (most recent call last): 2019-08-29 19:41:33,260 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:33,260 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:33,260 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:33,261 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:33,261 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:33,261 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:33,261 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 21:47706398-2147483647 -o /tmp/result.21:47706398-2147483647dvwcna.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:33,261 ERROR ------------------------------------------------------------ 2019-08-29 19:41:36,917 WARNING Cannot query reference sequence at :170806447-170806447 2019-08-29 19:41:36,918 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:36,918 ERROR ------------------------------------------------------------ 2019-08-29 19:41:36,918 ERROR Traceback (most recent call last): 2019-08-29 19:41:36,918 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:36,919 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:36,919 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:36,919 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:36,919 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:36,919 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:36,920 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 6:167912504-2147483647 -o /tmp/result.6:167912504-2147483647Z6NLeI.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:36,920 ERROR ------------------------------------------------------------ 2019-08-29 19:41:37,303 WARNING Cannot query reference sequence at :190214677-190214677 2019-08-29 19:41:37,303 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:37,304 ERROR ------------------------------------------------------------ 2019-08-29 19:41:37,304 ERROR Traceback (most recent call last): 2019-08-29 19:41:37,304 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:37,305 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:37,305 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:37,305 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:37,305 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:37,305 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:37,306 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 4:188114755-2147483647 -o /tmp/result.4:188114755-2147483647Pb56lZ.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:37,306 ERROR ------------------------------------------------------------ 2019-08-29 19:41:38,527 WARNING Cannot query reference sequence at :138401849-138401849 2019-08-29 19:41:38,527 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:38,527 ERROR ------------------------------------------------------------ 2019-08-29 19:41:38,527 ERROR Traceback (most recent call last): 2019-08-29 19:41:38,528 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:38,528 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], *arg[1]['kwargs']) 2019-08-29 19:41:38,528 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:38,528 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:38,528 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:38,529 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:38,529 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 9:136762355-139076411 -o /tmp/result.9:136762355-13907641113yu5u.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:38,529 ERROR ------------------------------------------------------------ 2019-08-29 19:41:39,733 WARNING Cannot query reference sequence at :139079107-139079107 2019-08-29 19:41:39,733 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:39,733 ERROR ------------------------------------------------------------ 2019-08-29 19:41:39,733 ERROR Traceback (most recent call last): 2019-08-29 19:41:39,733 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:39,733 ERROR return arg[1]['fun'](arg[0], arg[1]['args'], arg[1]['kwargs']) 2019-08-29 19:41:39,733 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:39,734 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:39,734 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:39,734 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:39,734 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 9:139076412-2147483647 -o /tmp/result.9:139076412-2147483647zYY_Q2.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:39,734 ERROR ------------------------------------------------------------ 2019-08-29 19:41:41,171 WARNING Cannot query reference sequence at :145147006-145147006 2019-08-29 19:41:41,171 ERROR Exception when running <function xcmpWrapper at 0x7f689d96ac08>: 2019-08-29 19:41:41,171 ERROR ------------------------------------------------------------ 2019-08-29 19:41:41,171 ERROR Traceback (most recent call last): 2019-08-29 19:41:41,171 ERROR File "/opt/hap.py/lib/python27/Tools/parallel.py", line 72, in parMapper 2019-08-29 19:41:41,172 ERROR return arg[1]['fun'](arg[0], *arg[1]['args'], **arg[1]['kwargs']) 2019-08-29 19:41:41,172 ERROR File "/opt/hap.py/lib/python27/Haplo/xcmp.py", line 70, in xcmpWrapper 2019-08-29 19:41:41,172 ERROR subprocess.check_call(to_run, shell=True, stdout=tfo, stderr=tfe) 2019-08-29 19:41:41,172 ERROR File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2019-08-29 19:41:41,172 ERROR raise CalledProcessError(retcode, cmd) 2019-08-29 19:41:41,172 ERROR CalledProcessError: Command 'xcmp /tmp/truth.ppxP6DiY.vcf.gz /tmp/query.ppoL4bN0.vcf.gz -l 8:143333332-2147483647 -o /tmp/result.8:143333332-2147483647MSsCPu.bcf -r /input/Homo_sapiens.GRCh38.dna.primary_assembly.fa -f 0 -n 16768 --expand-hapblocks 30 --window 50 --no-hapcmp 0 --qq QUAL' returned non-zero exit status 1 2019-08-29 19:41:41,172 ERROR ------------------------------------------------------------ 2019-08-29 19:41:45,982 ERROR One of the xcmp jobs failed. 2019-08-29 19:41:45,982 ERROR Traceback (most recent call last): 2019-08-29 19:41:45,982 ERROR File "/opt/hap.py/bin/hap.py", line 508, in 2019-08-29 19:41:45,983 ERROR main() 2019-08-29 19:41:45,983 ERROR File "/opt/hap.py/bin/hap.py", line 431, in main 2019-08-29 19:41:45,983 ERROR raise Exception("One of the xcmp jobs failed.") 2019-08-29 19:41:45,983 ERROR Exception: One of the xcmp jobs failed.

With vcfeval: docker run -v "${INPUT_DIR}":"/input" -v "${OUTPUT_DIR}:/output" pkrusche/hap.py /opt/hap.py/bin/hap.py /input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2_highconf.vcf.gz /output/output.vcf.gz -f "/input/NA12878_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-Solid_ALLCHROM_v3.2.1_highconf.bed" -r "/input/Homo_sapiens.GRCh38.dna.primary_assembly.fa" -o "/output/happy.output" --no-leftshift --no-decompose --engine=vcfeval --engine-vcfeval-template "/input/GRCh38_hs38d1.sdf" 2019-08-29 19:46:59,817 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. 2019-08-29 19:46:59,824 WARNING No reference file found at default locations. You can set the environment variable 'HGREF' or 'HG19' to point to a suitable Fasta file. [W] overlapping records at 1:30934817 for sample 0 [W] Variants that overlap on the reference allele: 328 [I] Total VCF records: 4154255 [I] Non-reference VCF records: 4154255 [I] Total VCF records: 684797 [I] Non-reference VCF records: 341918 2019-08-29 19:49:51,018 ERROR Error running rtg tools / vcfeval. Return code was 1, output: / Error: There were no sequence names in common between the reference and the supplied variant sets. Check the correct reference is being used.

2019-08-29 19:49:51,019 ERROR Traceback (most recent call last): 2019-08-29 19:49:51,019 ERROR File "/opt/hap.py/bin/hap.py", line 508, in 2019-08-29 19:49:51,019 ERROR main() 2019-08-29 19:49:51,019 ERROR File "/opt/hap.py/bin/hap.py", line 452, in main 2019-08-29 19:49:51,019 ERROR tempfiles += Haplo.vcfeval.runVCFEval(args.vcf1, args.vcf2, output_name, args) 2019-08-29 19:49:51,019 ERROR File "/opt/hap.py/lib/python27/Haplo/vcfeval.py", line 129, in runVCFEval 2019-08-29 19:49:51,019 ERROR raise Exception("Error running rtg tools / vcfeval. Return code was %i, output: %s / %s \n" % (rc, o, e)) 2019-08-29 19:49:51,019 ERROR Exception: Error running rtg tools / vcfeval. Return code was 1, output: / Error: There were no sequence names in common between the reference and the supplied variant sets. Check the correct reference is being used.

It says there were no sequence names in common between reference and supplied variant sets but I used the same reference for variant calling.

docker run -v "${INPUT_DIR}":"/input" -v "${OUTPUT_DIR}:/output" gcr.io/deepvariant-docker/deepvariant:"${BIN_VERSION}" /opt/deepvariant/bin/run_deepvariant --model_type=WES --ref=/input/Homo_sapiens.GRCh38.dna.primary_assembly.fa --reads=/input/SRR2635048_1.RG.sorted.bam --output_vcf=/output/output.vcf.gz --output_gvcf=/output/output.g.vcf.gz --num_shards=30

Is it something I am doing wrong?

Hi did you use GATK4 for the variant calling. You can try this command --no-leftshift --no-decompose --engine=vcfeval --gender=none

Lenbok commented 5 years ago

Usually that vcfeval message means your VCFs are using chromosome names like "1", "2", etc but the reference SDF has names like "chr1", "chr2", etc (or vice versa). Note that one of the warning message is using a coordinate like "1:30934817" (i.e. no "chr" prefix) but GRCh38_hs38d1 usually has names that are prefixed with "chr". I think happy does some level of converting name conversion, but I am not sure of the details, your best bet is to make sure the input files and SDF are all using the same naming convention.

HuTaoJiaZiAckerman commented 1 year ago

May be you solve this problem by installing rgt-tools using below the command.

conda install -c bioconda rtg-tools

Of cause! It also could be not work!