Demo codes and files?

[quevedor2]

1. bamgineer/demo_codes/wes_example.sh is a blank file

2. I was also wondering if there are any demo files to try out bamgineer with? The directory bamgineer/tcga_experiments/inputs/ only contains bed files of genes. As per the scripts used in "bamgineer/tcga_experiments/scripts/generation/T1", I can not find any of the files mentioned: "inputs/vcfs/normal_varscan.vcf", "inputs/bams/Normal.bam" or "inputs/beds/exons.bed"

[soroushsamadian]

Dear Rene,

Thank you for your interest in Bamgineer. Due to some major redesign and improvements, the new version of Bamgineer takes slightly different input arguments. As such I have reorganized the 'demo_codes' folder into "examples" folder where I have added detailed data preparation steps for a sample run of Bamgineer from sequencing data downloaded from Personal Genome Project website. Please note that this example will mostly serve as a "dry-run" rather than a full-blown evaluation (due to unknown parameters about the input files from Personal Genome).

Additionally due to size limitations, we are not able to share the Bam file on the github and you would still need to download the Bam files from the website. It's also recommended that you sort the Bam files by name (using specific naming convention explained) prior to running the program. Please also note that the paths in example config file may vary based on the specific environment.

That said, I would still highly recommend running Bamgineer on your own normal Bam file (if possible) whose specific parameters (such as target regions and so on) are known. The normal_vcf.het file can also be generated using a variety of tools such as HaplotypeCaller, or Varscan prior to the run.

Best,

Soroush

[quevedor2]

The "input_preperation.md" (typo, it's input_prepAration.md") has some minor typos and errors, I'll upload a revised version of this once I have everything piped together.

Could also be useful to list all the dependencies and versioning that you need. From my experience, this is how I have things set up:

module load samtools/1.2
module load tabix/0.2.6
module load bedtools/2.26.0
module load vcftools/0.1.12
module load sambamba/0.5.4
module load python/2.7

pip install virtualenv --user

virtualenv bamgineer_test
source bamgineer_test/bin/activate
pip install pysam==0.8.4 --user
pip install pyvcf --user
pip install pybedtools --user
pip install pathos --user

bamgineer/examples/scripts/run_example1.sh issues:

• why use so many error prone relative "../../scr/simulate.py" paths? Why not just do something like "/path/to/git/bamgineer/examples/scripts/run_example1.sh" or even just set a tunable variable like ${PDIR}/simulate.py
• also, your relative path is linking the "scr" dir and not the correct "src" dir
• "-cnv_bed ../inputs/cnv.bed"... does it accept a --cnv_bed argument? Your usage says it only accepts --cnv_amp or --cnv_del
• I'm guessing the "-c" argument is the short version of "--config"? You make no mention of this anywhere
• Also, your "-c" argument points to "config.bed" rather than "config.cfg"
• The config.cfg file that you provided in examples (untouched) throws this error "ConfigParser.MissingSectionHeaderError: File contains no section headers.".
• First line of config.cfg is "TER]".. not sure what its suppose to say, but I know it's missing an opening bracket
• Fixing the line endings of the config.cfg allows it to run into another error: "TypeError: sequence item 1: expected string, NoneType found". Stuck at this point, any advice on how to proceed?

README.md says that -cancer is an argument, but simulate.py says it's -cancertype (correct one is -cancertype)

• Adding a -cancertype LUAC argument seems to address the NoneType error found in handlers.py where it tries:

  cancer_type = params.GetCancerType()
  cancer_dir_path = "/".join([results_path, cancer_type])

  Is there a validation check in place or a default value you set anywhere to cancer_type to ensure that it tries to join something? It seems like cancer_type is a mandatory variable you need to set.

• using -cancertype LUAC, i get the new error:

  ___ phasing vcf file ___ 
  Traceback (most recent call last):
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/simulate.py", line 56, in <module>
  main(args)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/simulate.py", line 31, in main
  run_pipeline(results_path)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/methods.py", line 512, in run_pipeline
  initialize(results_path,haplotype_path,cancer_dir_path)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/methods.py", line 43, in initialize
  phaseVCF(vcf_path, phasedvcf)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/utils.py", line 30, in phaseVCF
  gzipFile(vcfpath)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/utils.py", line 51, in gzipFile
  with open(filename, 'rb') as f_in, gzip.open(filename+'.gz', 'wb') as f_out:
  IOError: [Errno 2] No such file or directory: 'normal_het.vcf'

  which can be solved by changing the config.cfg file to use absolute paths for normal_het.vcf and exons.bed instead of just listing the file name. May be worth it to set config.cfg as /path/to/normal_het.vcf as just listing normal_het.vcf implies the script knows the directory structures and paths needed to self-detect normal_het.vcf?

• After fixing the config.cfg file, I get the error:

  Traceback (most recent call last):
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/simulate.py", line 56, in <module>
  main(args)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/simulate.py", line 31, in main
  run_pipeline(results_path)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/methods.py", line 512, in run_pipeline
  initialize(results_path,haplotype_path,cancer_dir_path)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/methods.py", line 44, in initialize
  getVCFHaplotypes(phasedvcf, hap1vcf, hap2vcf)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/utils.py", line 91, in getVCFHaplotypes
  vcfh = gzip.GzipFile(phasedvcf, 'rb')
  File "/mnt/work1/software/python/2.7/lib/python2.7/gzip.py", line 89, in __init__
  fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
  IOError: [Errno 2] No such file or directory: '../outputs/normal_het_phased.vcf.gz'

  Seems like there's no check to make sure the utils.py/phaseVCF() function actually phases the VCF correctly?

• Seems like there's a problem in phasedVCF() when it runs the command:

  /mnt/work1/software/java/8/jdk1.8.0_45/bin/java -Xmx4g -jar \
  /mnt/work1/users/pughlab/projects/Benchmarking/Beagle/beagle.09Nov15.d2a.jar \
  gt=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/inputs/normal_het.vcf.gz \
  out=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/normal_het_phased

where it throws the error for beagle.log

Exception in thread "main" java.lang.IllegalArgumentException: nSamples==0
    at h.I.<init>(Unknown Source)
    at h.e.<init>(Unknown Source)
    at h.G.a(Unknown Source)
    at h.G$$Lambda$4/1880587981.apply(Unknown Source)
    at h.G.a(Unknown Source)
    at h.G$$Lambda$5/932172204.apply(Unknown Source)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
    at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:747)
    at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:721)
    at java.util.stream.AbstractTask.compute(AbstractTask.java:316)
    at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
    at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
    at java.util.stream.ReduceOps$ReduceOp.evaluateParallel(ReduceOps.java:714)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at h.G.c(Unknown Source)
    at h.G.a(Unknown Source)
    at main.Main.main(Unknown Source)

I think it's likely because the VCF file that you processed based on your "input_preperation.md" guide does not have the sample information in its final column as you would expect for vcfs; which is sort of expected when you recode a vcf...

#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO
chr21   9414112 .       T       C       56      PASS    .       GT:GQ:GQX:DP:DPF:AD
chr21   9414172 .       T       C       125     PASS    .       GT:GQ:GQX:DP:DPF:AD
chr21   9414185 .       T       C       37      HighSNVSB       .       GT:GQ:GQX:DP:DPF:AD
chr21   9414193 .       T       C       88      PASS    .       GT:GQ:GQX:DP:DPF:AD
chr21   9414215 .       A       G       72      PASS    .       GT:GQ:GQX:DP:DPF:AD
chr21   9414283 .       T       C       239     PASS    .       GT:GQ:GQX:DP:DPF:AD

Just a bit confused where it looks like beagle is expecting a sample from the vcf "java.lang.IllegalArgumentException: nSamples==0", but you lose that sample when you recode it in vcfTools. Seems a bit wrong to try and phase a recoded vcf since you lose all the information... is this what you were meaning to do? Is vcfTools suppose to be run like this? Not quite sure what it's doing:

#9. Extract heterozygous SNPs (in step 7) that reside in the exons bed region
vcftools --vcf chr21_22_het_genome.vcf --bed exons.bed --out normal_het --recode

mv normal_het.recode.vcf normal_het.vcf

[soroushsamadian]

Hi Rene,

Thank you for the comments and suggestions. Please find below my comments to the issues:

Listing dependencies and python virtual environment: This is a very good suggestion and I will list the dependencies in the documentation. However for users who do not have access to high performance clusters (HPC), the “module load …” command cannot be readily used and as such they need to directly edit the configuration file to point to the executable path for the external tools. I am in the process of creating a python installable package that can directly be installed (e.g. using “pip install bamgineer”) and automatically install missing dependencies and their appropriate version.

Error-prone relative path: The use of relative path is mainly to make the code transferable to any user-specific environment and is more a matter of personal preference. Anyhow, once I create the installable package , there will be no need to have either relative or absolute paths for python files (e.g. simulate.py). I can change the path for input/output files into absolute paths to make them less error-prone (though I think it’s fairly clear to the users that they can use absolute paths if they prefer).

Use of one CNV bed file versus separate bed files for gain and loss events: I have recently updated the code in the new version to accept one bed file for both gain and loss events , where the last column of the bed file is the absolute copy number for the corresponding CNV event and hence will determine the CNV event (e.g. last column > 3 denotes gain events). The documentation has been updated to reflect the change.

Config file extension: Doesn’t really matter what extension you use and additionally the example default config file will not work anyways since it has sample paths to tools and input files that are specific to the run environment. However, I agree that “.bed” extension can conceptually be misleading so I changed the example script to reflect it.

Config file argument: I changed “-c” argument to “-config” for clarity

First line of config: Thanks for pointing out this type. It was meant to [CLUSTER] not [TER]. I fixed the typo.

Error regarding “cancertype”: I believe you were running the master branch. I tried all the recent changes on “integration” branch. I have made the necessary changes and merged them into master branch. The “cancertype” argument shouldn’t be required any more.

Beagle exception: I did not encounter the error you get while testing it. I will rerun the code to see if I can reproduce it. In the meantime, please pull the latest changes and try rerunning the code and let me know if you still get the error.

Thank you,

Soroush

[quevedor2]

Beagle exception: I still get this error in the updated version.

Following the instructions in input_preparation.md, I run the following commands to set up the vcf (note: the download for me gives the file 2816, not the huA2692-veritas-gVCF-4.2.vcf.bz2 that you list in your input_preparation.md script):

#!/bin/bash
#
#$ -cwd
module load tabix/0.2.6
module load vcftools/0.1.15

#wget https://my.pgp-hms.org/user_file/download/2816
##mv huA2692-veritas-gVCF-4.2.vcf.bz2 genome.vcf.bz2  # Original line in input_preparation.md
#mv 2816 genome.vcf.bz2

bzip2 -d genome.vcf.bz2
bgzip genome.vcf
tabix genome.vcf.gz

tabix genome.vcf.gz chr21 > chr21.vcf
tabix genome.vcf.gz chr21 > chr22.vcf

grep "0/1" chr21.vcf > chr21_het.vcf
grep "0/1" chr22.vcf > chr22_het.vcf

cat chr21_het.vcf chr22_het.vcf > chr21_22_het_genome.vcf

vcftools --vcf chr21_22_het_genome.vcf --bed exons.bed --out normal_het --recode

mv normal_het.recode.vcf normal_het.vcf

Using the normal_het.vcf in my inputs directory, I use the following beagle command (extracted directly from your utils.py phaseVCF() function by adding a print(command) statement just prior to the runCommand(command) execution)

/mnt/work1/software/java/8/jdk1.8.0_45/bin/java -Xmx4g \
-jar /mnt/work1/users/pughlab/projects/Benchmarking/Beagle/beagle.09Nov15.d2a.jar \
gt=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/inputs/normal_het.vcf.gz \
out=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/normal_het_phased \
2> beagle.log

Using this command, I don't get the outputs/phasedvcfdir/normal_het_phased.vcf.gz that is required for downstream processing. As far as I know, this is the only error message I get from beagle.log

#beagle.log
Exception in thread "main" java.lang.IllegalArgumentException: nSamples==0
    at h.I.<init>(Unknown Source)
    at h.e.<init>(Unknown Source)
    at h.G.a(Unknown Source)
    at h.G$$Lambda$4/1880587981.apply(Unknown Source)
    at h.G.a(Unknown Source)
    at h.G$$Lambda$5/932172204.apply(Unknown Source)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
    at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:747)
    at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:721)
    at java.util.stream.AbstractTask.compute(AbstractTask.java:316)
    at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
    at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
    at java.util.stream.ReduceOps$ReduceOp.evaluateParallel(ReduceOps.java:714)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at h.G.c(Unknown Source)
    at h.G.a(Unknown Source)
    at main.Main.main(Unknown Source)

normal_het_phased.warnings is empty normal_het_phased.log gives the following output:

#normal_het_phased.log
beagle.09Nov15.d2a.jar
Copyright (C) 2014-2015 Brian L. Browning
Enter "java -jar beagle.jar" for a summary of command line arguments.
Start time: 03:09 PM EST on 31 Jan 2018

Command line: java -Xmx3641m -jar beagle.jar
  gt=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/inputs/normal_het.vcf.gz
  out=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/normal_het_phased

No genetic map is specified: using 1 cM = 1 Mb

[quevedor2]

I don't think it's an issue of java versions, but I can try java 1.7 and java1.8

I got the beagle/vcf error using Java/1.8. I kept as much as possible consistent with the config.cfg provided in the examples/script directory.

java_path  = /mnt/work1/software/java/8/jdk1.8.0_45/bin/java

I would think that the beagle.log error:

Exception in thread "main" java.lang.IllegalArgumentException: nSamples==0

would suggest that there is something malformed about the vcf that was made

[soroushsamadian]

You're right, I saw that you're using the right Java version. I figured out the issue though. It's the step where we overlap "chr21_22_genom.vcf" with "exon.bed" using vcftools (step 9). The chr21_22_genom.vcf has 10 columns whereas the generated file will end up having 8 columns missing FORMAT and SAMPLE columns. That's why you get the "nSamples = 0" in Beagle output.

It might be easier to use "bedtools intersect" instead of vcftools for this step. Anyhow, I'll update the documentation to address this (example/inputs/input_het.vcf will also be updated to have the right number of columns)

[quevedor2]

Just fixed the issue, but don't know what to expect for in my output or how to interpret this.

VCF-Fix

The issue was that the vcf being fed into vcftools to be intersected with the exons.bed file was missing headers. Simple fix was to take the headers straight from the genome.vcf.gz file and place it on top of the chr21_22... vcf file

zcat genome.vcf.gz | grep "^#" > header.txt

cat header.txt chr21_het.vcf chr22_het.vcf > chr21_22_het_genome.vcf

vcftools --vcf chr21_22_het_genome.vcf --bed exons.bed --out normal_het --recode

mv normal_het.recode.vcf normal_het.vcf

Running the execution script after properly formatting the normal_het.vcf gave me the following output

(bamgineer_test)Badger:66969Mb [quever@samwise scripts]$ sh run_example1.sh 
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs
normal_het.vcf
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/inputs/normal_het.vcf
 ___ phasing vcf file ___ 
/mnt/work1/software/java/8/jdk1.8.0_45/bin/java -Xmx4g -jar /mnt/work1/users/pughlab/projects/Benchmarking/Beagle/beagle.09Nov15.d2a.jar gt=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/inputs/normal_het.vcf.gz out=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/normal_het_phased 2> beagle.log
beagle.09Nov15.d2a.jar
Copyright (C) 2014-2015 Brian L. Browning
Enter "java -jar beagle.jar" for a summary of command line arguments.
Start time: 06:29 PM EST on 31 Jan 2018

Command line: java -Xmx3641m -jar beagle.jar
  gt=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/inputs/normal_het.vcf.gz
  out=/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/normal_het_phased

No genetic map is specified: using 1 cM = 1 Mb

reference samples:       0
target samples:          1

VCFtools - v0.1.12
(C) Adam Auton and Anthony Marcketta 2009

Parameters as interpreted:
    --vcf /mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/hap1_het.vcf
    --thin 50
    --out /mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/hap1_het_filtered
    --recode

After filtering, kept 0 out of 0 Individuals
Outputting VCF file...
After filtering, kept 0 out of a possible 0 Sites
File does not contain any sites
Run Time = 0.00 seconds

VCFtools - v0.1.12
(C) Adam Auton and Anthony Marcketta 2009

Parameters as interpreted:
    --vcf /mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/hap2_het.vcf
    --thin 50
    --out /mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/hap2_het_filtered
    --recode

After filtering, kept 0 out of 0 Individuals
Outputting VCF file...
After filtering, kept 0 out of a possible 0 Sites
File does not contain any sites
Run Time = 0.00 seconds

Output Question

I don't know how to interpret the output though. This is the tree structure I get after running the sample script

../outputs
├── cnv_dir
│   └── gain.bed
├── LUAC
│   ├── finalbams
│   ├── haplotypedir
│   │   ├── chr21_exons_in_roigain.bed
│   │   ├── exons_in_roigain.bed
│   │   └── het_snpgain.bed
│   └── logs
│       └── debug.log
└── phasedvcfdir
    ├── hap1_het_filtered.bed
    ├── hap1_het_filtered.log
    ├── hap1_het_filtered.recode.vcf
    ├── hap1_het.vcf
    ├── hap2_het_filtered.bed
    ├── hap2_het_filtered.log
    ├── hap2_het_filtered.recode.vcf
    ├── hap2_het.vcf
    ├── normal_het_phased.log
    ├── normal_het_phased.vcf.gz
    ├── normal_het_phased.warnings
    └── PHASED.BED

Can you help decipher this? I was expecting a bam file but didn't get it.

[soroushsamadian]

Thanks for the VCF fix Rene. Program will generate a series of bam files in "tmpbams" and "finalbams" which are self explanatory. But I suspect your code has finished immaturely due to a non-exising file or something.Can you have a look at the last few lines of the log file(logs/debug.log) and see if you can figure out?

[quevedor2]

This is the contents of debug.log, not sure what to look for. Anything stand out to you?

pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snp.bed.bed does not exist!
 ***** pipeline finished in 0.0 minutes ***** 

[soroushsamadian]

I suspect some of the created VCF files (in phasedvcfs directory) do not have variants in them( the ".bed.bed" extension in the log file should be fixed but I don't think that is causing the issue). Can you confirm they actually have variants in them? The empty VCF file will lead to blank snp_het file and hence the program will exit without creating the Bam files. It could be that Beagle is still not working the way it is supposed to and hence the phased_het.vcf maybe blank(?). I'll have a closer look at the demo example and let you know if I can figure it out.

On a different note, please remember to filter indels from your "normal_het.vcf" file prior to phasing. I will add this step to the documentation.

[soroushsamadian]

I had a closer looks and it seems that the normal_het.vcf file that is generated is not sorted. When I run Beagle I get the error "java.lang.IllegalArgumentException: markers not in chromosomal order". However this error will only be displayed on the console and not recorded in the log file. Running Beagle will generate phased_beagle.vcf that contains only the "header". So you need to sort the VCF (after removing Indels) and re-run the code.

It seems that your input VCF file has duplicated records (my guess is you have cat chr21 with itself instead of chr22, as chr22 has no variants in the input VCf file). Once you dedup the VCF (e.g. awk '!seen[$0]++' in.vcf > in_dedup.vcf), Beagle creates a non-empty phased file.

[quevedor2]

good catch; seems to stem from the input_preparation.md line: tabix genome.vcf.gz chr21 > chr22.vcf and bedtools merge -i chr22.gt50.bedgraphh > exons_chr22.bed

The exons.bed was only being made on chr21 due to the typo on .bedgraphh; and chr22.vcf was being created by tabix-selecting chr21.

I made those fixes, removed the indels, sorted the vcf, dedupped using the awk script (and double checked with a bash command), and still get no bam files in my output

## Pre-filtered line-count
(bamgineer_test)Badger:81318Mb [quever@samwise inputs]$ wc -l normal_het.vcf
13898 normal_het.vcf

## Awk Filter
(bamgineer_test)Badger:81570Mb [quever@samwise inputs]$ awk '!seen[$0]++' normal_het.vcf > in_dedup.vcf
(bamgineer_test)Badger:81318Mb [quever@samwise inputs]$ wc -l in_dedup.vcf 
13898 in_dedup.vcf

## Bash Filter
(bamgineer_test)Badger:81317Mb [quever@samwise inputs]$ sort in_dedup.vcf  | uniq | wc -l
13898

For your reference, these are the commands I've updated to using after remaking exons.bed

vcftools --vcf chr21_22_het_genome.vcf --bed exons.bed --out normal_het --recode
vcftools --vcf normal_het.recode.vcf --remove-indels --recode --recode-INFO-all --out snps_only
vcf-sort snps_only.recode.vcf > snps_only.recode.sort.vcf
awk '!seen[$0]++' snps_only.recode.sort.vcf > snps_only.recode.sort.dedup.vcf
mv snps_only.recode.sort.dedup.vcf normal_het.vcf

Running the command line version of beagle threw no java.lang.IllegalArgumentException: markers not in chromosomal order error for me, so I think that's working correctly. The normal_het_phased.vcf.gz contains information. Maybe it does have something to do with the missing/curious .bed.bed files? I'll tinker around with that and see if it addresses the issue

[soroushsamadian]

I think I found the bug (line 255 utils.py that assumed a specific format for cnv event files). Pushed the changes. Could you pull and retry please?

[quevedor2]

Looks like it's getting closer. After I pulled the new repo, the run_example1.sh script runs a bit longer before it gets bombarded with "Exception!"

Parameters as interpreted:
    --vcf /mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/hap2_het.vcf
    --thin 50
    --out /mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/phasedvcfdir/hap2_het_filtered
    --recode

After filtering, kept 0 out of 0 Individuals
Outputting VCF file...
After filtering, kept 5424 out of a possible 6905 Sites
Run Time = 0.00 seconds
Exception! 
Exception! 
Exception! 
...
Exception! 
Exception! 
Exception! 
Exception! 

The debug.log is as follows (not quite sure what I should be looking for here):

pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snp.bed.bed does not exist!
 ***** pipeline finished in 0.0 minutes ***** 
pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snp.bed.bed does not exist!
 ***** pipeline finished in 0.0 minutes ***** 
pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snp.bed.bed does not exist!
 ***** pipeline finished in 0.1 minutes ***** 
pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snp.bed.bed does not exist!
 ***** pipeline finished in 0.1 minutes ***** 
pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snp.bed.bed does not exist!
 ***** pipeline finished in 0.1 minutes ***** 
pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roi.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_exons_in_roi.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snp.bed.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr21_het_snp.bed.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snp.bed.bed does not exist!
 ***** pipeline finished in 0.1 minutes ***** 
pipeline started!
 --- Initializing input files  --- 
--- initialization complete ---
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_exons_in_roigain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_exons_in_roigain.bed does not exist!
find_roi_bam complete successfully for chr21gain
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr2_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr1_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr3_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr5_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr4_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr15_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr9_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr8_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr7_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr6_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr12_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr10_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr13_het_snpgain.bed does not exist!
___ Bamgineer main engine started ___
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr11_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr14_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr16_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr17_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr18_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr20_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr19_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chr22_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrX_het_snpgain.bed does not exist!
/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs/LUAC/haplotypedir/chrY_het_snpgain.bed does not exist!
+++ coverage ratio for: chr21_roigain.sorted.bam: 0.278378637301
not enough reads for chr21_roigain.sorted.bam
 ***** pipeline finished in 3.4 minutes ***** 

[soroushsamadian]

Never mind those "Exceptions"; they're printed out everytime there is a problem with an individual read (line 362 method.py). I will delete the print line as it really isn't informative. Perhaps I can write the read names for which the exception happens to a seperate file. However the line before the last (not enough reads for ...), shows that Bamgineer was not able to "re-pair" enough reads to generate new ones to obtain the desired CNV state (for CNV=3 Bamginner needed ~0.5 coverage for new reads compared to the old, but found 0.27 and hence complains). In that case, one needs to put a less stringent criteria on new paired read generation.

That said, I suspect there won't indeed be enough read for this example. I will try running it with the examples in the tutorial and see if I get the same result. Anyhow, I do recommend running it on the actual Bam/Bed/regions and ref files as for the the case explained here, I use a lot of assumption and as explained it's mainly to perform a dry-run test. Will keep you posted.

[quevedor2]

It's good to know that Bamgineer ran to completion correctly then!!

However, I'm not too clear on this answer (is BioRxiv article updated? is that the most recent version of this manuscript?)? When you say Bamgineer needs 0.5 coverage, are you talking about allelic fraction? Not sure what 0.5 coverage means... GATK DepthofCoverage shows that Chr21 and Chr22 are sequenced to an average of ~32x and ~36x respectively.

Also, not quite sure what you mean by "running it on the actual bam/bed/regions and ref files as for the case explained here". I ran it on the PGP chromosomes that you suggested if that's the case you were talking about?

[quevedor2]

Hi Soroush, can you comment on your last reply? Would it be feasible to run bamgineer on a PGP sample or is there just not enough coverage to run with your tool?

[soroushsamadian]

Hi Rene, as I mentioned in my data preparation document, The dataset used from PGP does not provide the target regions (i.e. exons) used to generate the Bam files, and as such I came up with an approximation approach to generate exon bed file by making a bed file from all contiguous regions above a certain coverage threshold. I suspect, this may cause the problem with coverage. That's why I did suggest, it's far more reliable to run Bamgineer on your costum Bam file with known target regions and so on.

Also w.r.t my comment about "X0.5" additional coverage, I meant since the input CNV bed file has the absolute CN of 3, the total number of newly generated reads will have to be 1.5 times of the original coverage(i.e ~1.5x32 for chr21 and 1.5x36 for chr22). So if all the steps are done correctly, the "not enough read" message from Bamgineer means that Bamgineer was unable to find enough reads(meeting the criteria such as insert-size and so on) to "re-pair" and regenerate new reads (needed 0.5 times total but found 0.28 according to message) .

Anyhow, I will still follow the step-by-step guide in data prep section and see if I can reproduce your coverage problem. Will keep you posted.

[quevedor2]

Hey Soroush,

I'm starting to look back into bamgineer and we intend on running bamgineer on PGP samples due to the large number of normals available to simulate different types of tumors. Do you suggest that there is not enough coverage in these samples to be usable for Bamgineer?

I've re-ran the pipeline with exons.bed using an absolute CN of 2 and one with an absolute CN of 1: For the absolute CN of 1, it throws a division by zero error in the debug.log:

Exception in find_roi_bam float division by zero
Traceback (most recent call last):
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/methods.py", line 499, in implement_cnv
    ratio_kept = float(countReads(bamsortfn)) / float(countReads(bamfn))
ZeroDivisionError: float division by zero
 ***** pipeline finished in 2.4 minutes ***** 

For the absolute CN of 2, it shouldnt need to add any new reads so we wouldn't run into the same error where not enough reads were present? However, I still get the division by zero error:

Exception in find_roi_bam float division by zero
Traceback (most recent call last):
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/methods.py", line 499, in implement_cnv
    ratio_kept = float(countReads(bamsortfn)) / float(countReads(bamfn))
ZeroDivisionError: float division by zero
 ***** pipeline finished in 2.5 minutes ***** 

In neither case do I get a bam outputted.

[soroushsamadian]

Hi Rene, Can you please provide the exact CNV file(s) you are using as the input?

[quevedor2]

PDIR='/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test'

[quever@samwise scripts]$ cat ${PDIR}/inputs/cnv.bed
chr21   30227447    47076809    1

[soroushsamadian]

Thanks, I will try to replicate the error and will let you know.

[soroushsamadian]

Hi Rene, I tested the code and there was a typo in the loss function. Please pull from master branch and try re-running the script. Please note that I commented out the line where Bamgineer deleted temporary bam files "..outputs/tmpbams/" for debugging purposes. The folder can get too large and should be deleted (line 623 in methods.py).

Regarding CN=2 (LOH) event, I will perform some additional tests and commit the latest code within the next couple of days. Once tested I will add back the commented line to delete tmpbams folder. Anyhow, you should now see the output bam file generated in the "finalbams" directory.

Soroush

[quevedor2]

Just a note, your recent "tidying up" commit (27d3184f0c0035a64c60bc08c0080955879497bf) broke your script. You removed importing handle from handlers, but you still call the object on line 562 of methods.py

/mnt/work1/users/pughlab/projects/shallowWGS_benchmark/bamgineer_test/outputs
Traceback (most recent call last):
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/simulate.py", line 69, in <module>
    main(args)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/simulate.py", line 39, in main
    run_pipeline(results_path)
  File "/mnt/work1/users/home2/quever/git/bamgineer/src/methods.py", line 562, in run_pipeline
    haplotype_path, cancer_dir_path, tmpbams_path, finalbams_path, log_path, logfile = handle.GetProjectPaths(
NameError: global name 'handle' is not defined

[soroushsamadian]

Thanks Rene, I accidently removed some imports. FIxed :)

[soroushsamadian]

Also I think you should be able to run "gain" (CN=3) without a problem (just tested and the number of reads seems to be fine)

[quevedor2]

Unfortunately, with the latest pull, I can not run the "gain" like you did?

When I run the gain cnv bed: chr21 30227447 47076809 3; I still fall short on coverage ratio and end up with no final bams

+++ coverage ratio for: chr21_roigain.sorted.bam: 0.382933236524
not enough reads for chr21_roigain.sorted.bam
 ***** pipeline finished in 3.6 minutes ***** 

The only one I do get a final bam for is the loss cnv bed, chr21 30227447 47076809 1. Since I assume we are using the same PGP sample (huA2692E) and working on the same chromosomes (Chr21, Chr22) and all other parameters being the same, I would think that you just selected a different region of the genome that has adequate coverage for it to simulate a gain? What bed did you use to simulate your gain? Also, how did you test the number of reads, is it just a samtools view with a wc -l?

Just to clarify, did you clean up your output folder after running the "loss" (CN=1) script before running the "gain" (CN=3) script? I've noticed that if you run them sequentially, then the "gain" (CN=3) will leverage existing temporary files and reproduce the preceeding runs results (maybe should be addressed with run-specific temporary identifiers that get cleaned at the end?)

[quevedor2]

Hey soroush, Congrats on the paper acceptance! Following up on this again.

• What bed did you use to simulate your gain?
• Also, how did you test the number of reads, is it just a samtools view with a wc -l?
• did you clean up your output folder after running the "loss" (CN=1) script before running the "gain" (CN=3) script?

I'll add a fix shortly for creating run-specific temporary identifiers and cleaning up between runs to prevent reuse of existing files.

[soroushsamadian]

Hey Rene, Thank you for the congratulations and also thank for your support and contributions to make Bamgineer better; much appreciated. Here's the answer to your questions :

Bed file used : chr21 30227447 47076809 3 Final number of reads: 9047080 (note that it the final bam file should be named test.bam as indicated by the run script. So if you're not getting test.bam in the output, it means probably the final merge hasn't taken place) output cleaned? : Yes, I ran CN=3 from scratch on a different folder

Seems like we both used the same CNV region; so it's strange that you didn't get enough reads for that region. Have you pulled the latest version?

Hope it helps, Soroush

[quevedor2]

After checking over some of your inputs, it seems like your exons.bed (specified in your config file) wasn't made according to your specifications outlined in step 8. Create "exons.bed" file in input_preparation.md. Can you double check your pipeline for this and try again?

For reference, these are the md5sums for our input files for Step 8:

# My md5sums
3eb30670cfa3dd184931d8e27480bf6f`  chr21.bam
5fa9d9defca46111fe5c6dbf4cdaa302  chr22.bam
434f950f81ebd1a77fecff0c0154550a  bedit.sh

# Your md5sums
3eb30670cfa3dd184931d8e27480bf6f  chr21.bam
5fa9d9defca46111fe5c6dbf4cdaa302  chr22.bam
434f950f81ebd1a77fecff0c0154550a  bedit.sh

However, when I compared my exons.bed to your exons.bed, this is what I got:

# My exons.bed
md5sums:     38f824cc251c50036f2ef3c1af56e7ee  exons.bed
line-count:    47399 exons.bed
size:              1134589 exons.bed

# Your exons.bed (also includes Chr1?)
md5sums:     785f73d276920ea82f2c9dad2cddb44f  exons.bed
line-count:    216812 exons.bed
size:              5182092 Feb 26 15:08 exons.bed

Extra note: I didn't think there any kind of random number generator going on with something like bedtools, but I recreated my exons.bed file twice and got the same md5sums each time.