testing my installation

[MurliNair]

Hi, I am getting the following error when I test the Cancer tumor normal example. I think there are two main errors 1> "Aligning lane 1_2014-08-13_dream-syn3 with bwa aligner cat: write error: Broken pipe" 2>" [E::bwa_idx_load_from_disk] fail to locate the index files ' returned non-zero exit status 1 " It looks like I have not provided the bwa index file. I am not sure where to put the index file, if I create one for Hg19. Could you comment on that please, and anything else I may have missed? Thanks ../Murli

===============================================

$> bcbio_nextgen.py ../config/cancer-dream-syn3.yaml -n 8 [2017-06-03T19:53Z] System YAML configuration: /mnt/workspaceXFS/local/share/bcbio/galaxy/bcbio_system.yaml [2017-06-03T19:53Z] Resource requests: bwa, sambamba, samtools; memory: 3.00, 3.00, 3.00; cores: 16, 16, 16 [2017-06-03T19:53Z] Configuring 1 jobs to run, using 8 cores each with 24.1g of memory reserved for each job [2017-06-03T19:53Z] Timing: organize samples [2017-06-03T19:53Z] multiprocessing: organize_samples [2017-06-03T19:53Z] Using input YAML configuration: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/config/cancer-dream-syn3.yaml [2017-06-03T19:53Z] Checking sample YAML configuration: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/config/cancer-dream-syn3.yaml [2017-06-03T19:53Z] Testing minimum versions of installed programs [2017-06-03T19:53Z] Timing: alignment preparation [2017-06-03T19:53Z] multiprocessing: prep_align_inputs [2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-03T19:53Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-03T19:53Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-03T19:53Z] multiprocessing: disambiguate_split [2017-06-03T19:53Z] Timing: alignment [2017-06-03T19:53Z] multiprocessing: process_alignment [2017-06-03T19:53Z] Aligning lane 1_2014-08-13_dream-syn3 with bwa aligner cat: write error: Broken pipe [2017-06-03T19:53Z] bwa mem alignment from fastq: syn3-normal [2017-06-03T19:53Z] [E::bwa_idx_load_from_disk] fail to locate the index files [2017-06-03T19:53Z] Uncaught exception occurred Traceback (most recent call last): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 22, in run _do_run(cmd, checks, log_stdout, env=env) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 102, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/bwa mem -c 250 -M -t 8 -R '@RG\tID:syn3-normal\tPL:illumina\tPU:1_2014-08-13_dream-syn3\tSM:syn3-normal' -v 1 /mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37 <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_1.fq.gz 1 20000000) <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_2.fq.gz 1 20000000) | /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/samtools sort -n -@ 8 -m 1G -O bam -T /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000-sorttmp-namesort -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000.bam - [E::bwa_idx_load_from_disk] fail to locate the index files ' returned non-zero exit status 1 Traceback (most recent call last): File "/home/mnair/local/bin/bcbio_nextgen.py", line 234, in main(kwargs) File "/home/mnair/local/bin/bcbio_nextgen.py", line 43, in main run_main(kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 42, in run_main fc_dir, run_info_yaml) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 86, in _run_toplevel for xs in pipeline(config, run_info_yaml, parallel, dirs, samples): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 126, in variant2pipeline samples = run_parallel("process_alignment", samples) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel return run_multicore(fn, items, config, parallel=parallel) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore for data in joblib.Parallel(parallel["num_jobs"], batch_size=1)(joblib.delayed(fn)(x) for x in items): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 804, in call while self.dispatch_one_batch(iterator): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 662, in dispatch_one_batch self._dispatch(tasks) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 570, in _dispatch job = ImmediateComputeBatch(batch) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 183, in init self.results = batch() File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in call return [func(*args, *kwargs) for func, args, kwargs in self.items] File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 50, in wrapper return apply(f, args, *kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 93, in process_alignment return sample.process_alignment(args) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/sample.py", line 116, in process_alignment data = align_to_sort_bam(fastq1, fastq2, aligner, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/alignment.py", line 65, in align_to_sort_bam names, align_dir, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/alignment.py", line 119, in _align_from_fastq out = align_fn(fastq1, fastq2, align_ref, names, align_dir, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/ngsalign/bwa.py", line 156, in align_pipe names, rg_info, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/ngsalign/bwa.py", line 167, in _align_mem [do.file_nonempty(tx_out_file), do.file_reasonable_size(tx_out_file, fastq_file)]) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 22, in run _do_run(cmd, checks, log_stdout, env=env) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 102, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) subprocess.CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/bwa mem -c 250 -M -t 8 -R '@RG\tID:syn3-normal\tPL:illumina\tPU:1_2014-08-13_dream-syn3\tSM:syn3-normal' -v 1 /mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37 <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_1.fq.gz 1 20000000) <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_2.fq.gz 1 20000000) | /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/samtools sort -n -@ 8 -m 1G -O bam -T /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000-sorttmp-namesort -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmp9d3OlW/syn3-normal-sort-1_20000000.bam - [E::bwa_idx_load_from_disk] fail to locate the index files ' returned non-zero exit status 1

[chapmanb]

Murli; Thanks for testing out bcbio and for the detailed report. The cat: write error: Broken pipe message is a non-problematic output from taking head on a file (and thus avoiding reading the entire file) and I pushed a fix to avoid having this confusing message printed.

The actual issue you're running into is that it doesn't appear you've install bwa index files. You can do this with:

bcbio_nextgen.py upgrade --data --aligners bwa

which will add bwa indices for all the genomes you previously installed:

http://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html#upgrade

Hope this helps get things running cleanly. Please let us know if you have any other issues.

[MurliNair]

Hi, Thanks a lot for your reply. It still does not seem to have found the index files. I have two outputs that I am appending here, upgrade and the run. Appreciate your help. Cheers../Murli

The following is the output for the upgrade.

========================================================== [mnair@bioinformatics genomeAnalysisTools]$ bcbio_nextgen.py upgrade --data --aligners bwa Upgrading bcbio Upgrading bcbio-nextgen data files --2017-06-05 08:31:56-- https://github.com/chapmanb/cloudbiolinux/archive/master.tar.g z Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112 Connecting to github.com (github.com)|192.30.253.113|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following] --2017-06-05 08:31:57-- https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/mast er Resolving codeload.github.com (codeload.github.com)... 192.30.253.121, 192.30.253.120 Connecting to codeload.github.com (codeload.github.com)|192.30.253.121|:443... connecte d. HTTP request sent, awaiting response... 200 OK Length: unspecified [application/x-gzip] Saving to: 'STDOUT'

 0K ........ ........ ........ ........ ........ ........ 6.79M

3072K ........ ........ ........ ...... 12.9M=0.6s

2017-06-05 08:31:58 (8.32 MB/s) - written to stdout [5163464]

mv: cannot stat 'master': No such file or directory Setting up virtual machine INFO: <cloudbio.flavor.Flavor instance at 0x7efcad10bd40> INFO: This is a ngs_pipeline_minimal flavor INFO: Reading default fabricrc.txt DBG [config.py]: Using config file /mnt/workspaceXFS/mnair/research/CancerGenomics/geno meAnalysisTools/tmpbcbio-install/cloudbiolinux/cloudbio/../config/fabricrc.txt INFO: Distribution auto INFO: Get local environment INFO: CentOS setup DBG [distribution.py]: NixPkgs: Ignored [localhost] local: echo $HOME [localhost] local: uname -m INFO: Now, testing connection to host... INFO: Connection to host appears to work! DBG [utils.py]: Expand paths INFO: List of genomes to get (from the config file at '{'install_liftover': False, 'gen ome_indexes': ['bwa', 'bowtie2', 'rtg'], 'genomes': [{'name': 'Human (GRCh37)', 'valida tion': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'dream-syn3', 'dream-syn4'], 'a nnotations': ['GA4GH_problem_regions', 'capture_regions', 'MIG', 'prioritize', 'dbsnp', 'hapmap', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', 'clinvar', 'cosmic', 'ancestral', 'qsignature', 'genesplicer', 'effects_transcripts', 'vcfanno', ' viral', 'transcripts', 'RADAR', 'mirbase'], 'dbkey': 'GRCh37', 'indexes': ['seq']}], 'i nstall_uniref': False}'): Human (GRCh37) bcbio-nextgen data upgrade complete. Upgrade completed successfully.

The following is the output testing the installation

================================================= [mnair@bioinformatics work]$ bcbio_nextgen.py ../config/cancer-dream-syn3.yam l -n 8 [2017-06-05T12:35Z] System YAML configuration: /mnt/workspaceXFS/local/share/ bcbio/galaxy/bcbio_system.yaml [2017-06-05T12:35Z] Resource requests: bwa, sambamba, samtools; memory: 3.00, 3.00, 3.00; cores: 16, 16, 16 [2017-06-05T12:35Z] Configuring 1 jobs to run, using 8 cores each with 24.1g of memory reserved for each job [2017-06-05T12:35Z] Timing: organize samples [2017-06-05T12:35Z] multiprocessing: organize_samples [2017-06-05T12:35Z] Using input YAML configuration: /mnt/workspaceXFS/mnair/r esearch/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/con fig/cancer-dream-syn3.yaml [2017-06-05T12:35Z] Checking sample YAML configuration: /mnt/workspaceXFS/mna ir/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3 /config/cancer-dream-syn3.yaml [2017-06-05T12:35Z] Testing minimum versions of installed programs [2017-06-05T12:35Z] Timing: alignment preparation [2017-06-05T12:35Z] multiprocessing: prep_align_inputs [2017-06-05T12:35Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-05T12:35Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-05T12:35Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-05T12:35Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-05T12:35Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-05T12:35Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-05T12:35Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-05T12:35Z] Configuring 2 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-05T12:35Z] multiprocessing: disambiguate_split [2017-06-05T12:35Z] Timing: alignment [2017-06-05T12:35Z] multiprocessing: process_alignment [2017-06-05T12:35Z] Aligning lane 1_2014-08-13_dream-syn3 with bwa aligner cat: write error: Broken pipe [2017-06-05T12:35Z] bwa mem alignment from fastq: syn3-normal [2017-06-05T12:35Z] [E::bwa_idx_load_from_disk] fail to locate the index file s [2017-06-05T12:35Z] Uncaught exception occurred Traceback (most recent call last): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/provenance/do.py", line 22, in run _do_run(cmd, checks, log_stdout, env=env) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/provenance/do.py", line 102, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && /mnt/workspa ceXFS/local/share/bcbio/galaxy/../anaconda/bin/bwa mem -c 250 -M -t 8 -R ' @RG\tID:syn3-normal\tPL:illumina\tPU:1_2014-08-13_dream-syn3\tSM:syn3-normal' -v 1 /mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37 <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisT ools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_synthetic_cha llenge_set3_normal_NGv3_1.fq.gz 1 20000000) <(grabix grab /mnt/workspaceXFS/m nair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-sy n3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv3_2.fq.gz 1 20000000) | /mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/samt ools sort -n -@ 8 -m 1G -O bam -T /mnt/workspaceXFS/mnair/research/CancerGeno mics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmpMu8EY 6/syn3-normal-sort-1_20000000-sorttmp-namesort -o /mnt/workspaceXFS/mnair/res earch/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/ bcbiotx/tmpMu8EY6/syn3-normal-sort-1_20000000.bam - [E::bwa_idx_load_from_disk] fail to locate the index files ' returned non-zero exit status 1 Traceback (most recent call last): File "/home/mnair/local/bin/bcbio_nextgen.py", line 234, in main(kwargs) File "/home/mnair/local/bin/bcbio_nextgen.py", line 43, in main run_main(kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/pipeline/main.py", line 42, in run_main fc_dir, run_info_yaml) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/pipeline/main.py", line 86, in _run_toplevel for xs in pipeline(config, run_info_yaml, parallel, dirs, samples): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/pipeline/main.py", line 126, in variant2pipeline samples = run_parallel("process_alignment", samples) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/distributed/multi.py", line 28, in run_parallel return run_multicore(fn, items, config, parallel=parallel) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/distributed/multi.py", line 86, in run_multicore for data in joblib.Parallel(parallel["num_jobs"], batch_size=1)(joblib.de layed(fn)(x) for x in items): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/jo blib/parallel.py", line 804, in call while self.dispatch_one_batch(iterator): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/jo blib/parallel.py", line 662, in dispatch_one_batch self._dispatch(tasks) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/jo blib/parallel.py", line 570, in _dispatch job = ImmediateComputeBatch(batch) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/jo blib/parallel.py", line 183, in init self.results = batch() File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/jo blib/parallel.py", line 72, in call return [func(*args, *kwargs) for func, args, kwargs in self.items] File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/utils.py", line 50, in wrapper return apply(f, args, *kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/distributed/multitasks.py", line 93, in process_alignment return sample.process_alignment(args) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/pipeline/sample.py", line 116, in process_alignment data = align_to_sort_bam(fastq1, fastq2, aligner, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/pipeline/alignment.py", line 65, in align_to_sort_bam names, align_dir, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/pipeline/alignment.py", line 119, in _align_from_fastq out = align_fn(fastq1, fastq2, align_ref, names, align_dir, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/ngsalign/bwa.py", line 156, in align_pipe names, rg_info, data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/ngsalign/bwa.py", line 167, in _align_mem [do.file_nonempty(tx_out_file), do.file_reasonable_size(tx_out_file, fast q_file)]) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/provenance/do.py", line 22, in run _do_run(cmd, checks, log_stdout, env=env) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/provenance/do.py", line 102, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) subprocess.CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && / mnt/workspaceXFS/local/share/bcbio/galaxy/../anaconda/bin/bwa mem -c 250 -M -t 8 -R '@RG\tID:syn3-normal\tPL:illumina\tPU:1_2014-08-13_dream-syn3\tSM:s yn3-normal' -v 1 /mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/ seq/GRCh37 <(grabix grab /mnt/workspaceXFS/mnair/research/CancerGenomics/geno meAnalysisTools/bcbioExample/cancer-dream-syn3/work/align_prep/syn3-normal_sy nthetic_challenge_set3_normal_NGv3_1.fq.gz 1 20000000) <(grabix grab /mnt/wor kspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/canc er-dream-syn3/work/align_prep/syn3-normal_synthetic_challenge_set3_normal_NGv 3_2.fq.gz 1 20000000) | /mnt/workspaceXFS/local/share/bcbio/galaxy/../anacon da/bin/samtools sort -n -@ 8 -m 1G -O bam -T /mnt/workspaceXFS/mnair/research /CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbio tx/tmpMu8EY6/syn3-normal-sort-1_20000000-sorttmp-namesort -o /mnt/workspaceXF S/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream -syn3/work/bcbiotx/tmpMu8EY6/syn3-normal-sort-1_20000000.bam - [E::bwa_idx_load_from_disk] fail to locate the index files ' returned non-zero exit status 1 Exception in thread Thread-1 (most likely raised during interpreter shutdown) : Traceback (most recent call last): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/threading.py", l ine 801, in __bootstrap_inner File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/threading.py", l ine 754, in run File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/lo gbook/queues.py", line 276, in _target File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/lo gbook/queues.py", line 298, in dispatch_once File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bc bio/log/init.py", line 47, in recv File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/lo gbook/queues.py", line 565, in recv File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/multiprocessing/ queues.py", line 135, in get <type 'exceptions.EOFError'>:

[chapmanb]

Murli; Thanks for following up and apologies about the continued problems. It looks like bcbio thinks you installed the bwa indices already but something is wrong with the pointers to the indexes. You're getting a reference of:

/mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37

but it should be:

/mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/bwa/GRCh37.fa

Some questions/ideas for troubleshooting:

• How did you install bcbio initially? Have you done any tweaking after installation?
• Does the bwa directory exists and have indices in it? (/mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/bwa/)
• Do the index files in your installation point to the correct bwa location? These should be at: /mnt/workspaceXFS/local/share/bcbio/galaxy/tool-data/bwa_index.loc

Hope this helps.

[MurliNair]

Hi, Thanks again for your comments .

1. bcbio installed cleanly using the following wget https://raw.github.com/chapmanb/bcbio-nextgen/master/scripts/bcbio_nextgen_install.py python bcbio_nextgen_install.py ~/local/share/bcbio --tooldir=~/local --genomes GRCh37 --aligners bwa --aligners bowtie2

2. I did not tweak anything manually.

3. Yes, the bwa directory exists here (/mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/bwa/) It has the following files GRCh37.fa.amb GRCh37.fa.bwt GRCh37.fa.sa GRCh37.fa.ann GRCh37.fa.pac

4. I have following files in this location (/mnt/workspaceXFS/local/share/bcbio/galaxy/tool-data) bowtie2_indices.loc gatk_sorted_picard_index.loc sam_fa_indices.loc bwa_index.loc picard_index.loc

5. I have the following files in this location (/mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq) GRCh37-resources.yaml GRCh37.fa GRCh37.fa.gz GRCh37.fa.gz.gzi GRCh37.dict GRCh37.fa.fai GRCh37.fa.gz.fai

I don't want to manually tweak anything as I am not familiar with bcbio yet and hence all the questions. Let me know if the file structure looks okay to you or I need to move anything around. Cheers../Murli

[MurliNair]

Hi, I reinstalled everything all over using the command given in the "Automated" section. It installed cleanly. I then installed gatk using the following command gatk-register GenomeAnalysisTK-3.7-0.tar.bz2 I then ran bcbio_nextgen.py upgrade --data --aligners bwa

The cancer -normal did not run cleanly, gave the same error. Also I want to point out, that (/mnt/workspaceXFS/local/share/bcbio/galaxy/tool-data ) does not contain the index files now. It has the following files gatk_sorted_picard_index.loc picard_index.loc sam_fa_indices.loc

Did I miss anything? Kindly let me know. Thx a ton for your time. Cheers../Murli

[chapmanb]

Murli; Thanks for the updates and all the details about your setup. I've been working on reproducing and debugging the issue and believe I've isolated the issue and pushed a fix. If you remove any cached cloudbiolinux downloads and re-run an upgrade:

rm -rf tmpbcbio-install
bcbio_nextgen.py upgrade --data --aligners bwa

it should download the installation fix and hopefully correctly supply indices.

Apologies about the issue, this came from recent work to start generalizing the downloads but resulted in missing these indices which get installed from pre-build tarballs. Thank you for the help tracking it down and I hope with this in place everything runs smoothly for you.

[MurliNair]

Hi Brad, It is running fine, but its not completed yet. I shall keep you posted. Thank you for taking the time to helping me with this. I greatly appreciate it. Cheers../Murli

[MurliNair]

Brad: The test code is running but it sometimes throw Java exception as follows. Should I be worried about it?

[2017-06-06T23:44Z] Genotyping with varscan: ('2', 221341378, 243199373) syn3-normal-sort-2_221341378_243199373-prep.bam [2017-06-06T23:44Z] Varscan [2017-06-06T23:44Z] [mpileup] 1 samples in 1 input files [2017-06-06T23:44Z] Set max per-file depth to 8000 [2017-06-06T23:44Z] [mpileup] 1 samples in 1 input files [2017-06-06T23:44Z] Set max per-file depth to 8000 [2017-06-06T23:44Z] Normal Pileup: /dev/fd/63 [2017-06-06T23:44Z] Tumor Pileup: /dev/fd/62 [2017-06-06T23:44Z] Min coverage: 5x for Normal, 5x for Tumor [2017-06-06T23:44Z] Min reads2: 2 [2017-06-06T23:44Z] Min strands2: 1 [2017-06-06T23:44Z] Min var freq: 0.1 [2017-06-06T23:44Z] Min freq for hom: 0.75 [2017-06-06T23:44Z] Normal purity: 1.0 [2017-06-06T23:44Z] Tumor purity: 1.0 [2017-06-06T23:44Z] Min avg qual: 15 [2017-06-06T23:44Z] P-value thresh: 0.98 [2017-06-06T23:44Z] Somatic p-value: 0.05 [2017-06-06T23:44Z] Varscan [2017-06-06T23:44Z] [mpileup] 1 samples in 1 input files [2017-06-06T23:44Z] Set max per-file depth to 8000 [2017-06-06T23:44Z] [mpileup] 1 samples in 1 input files [2017-06-06T23:44Z] Set max per-file depth to 8000 [2017-06-06T23:44Z] Normal Pileup: /dev/fd/63 [2017-06-06T23:44Z] Tumor Pileup: /dev/fd/62 [2017-06-06T23:44Z] Min coverage: 5x for Normal, 5x for Tumor [2017-06-06T23:44Z] Min reads2: 2 [2017-06-06T23:44Z] Min strands2: 1 [2017-06-06T23:44Z] Min var freq: 0.1 [2017-06-06T23:44Z] Min freq for hom: 0.75 [2017-06-06T23:44Z] Normal purity: 1.0 [2017-06-06T23:44Z] Tumor purity: 1.0 [2017-06-06T23:44Z] Min avg qual: 15 [2017-06-06T23:44Z] P-value thresh: 0.98 [2017-06-06T23:44Z] Somatic p-value: 0.05 [2017-06-06T23:44Z] 1423488 positions in tumor [2017-06-06T23:44Z] 1423484 positions shared in normal [2017-06-06T23:44Z] 1401653 had sufficient coverage for comparison [2017-06-06T23:44Z] 1399504 were called Reference [2017-06-06T23:44Z] 0 were mixed SNP-indel calls and filtered [2017-06-06T23:44Z] 339 were removed by the strand filter [2017-06-06T23:44Z] 2074 were called Germline [2017-06-06T23:44Z] 5 were called LOH [2017-06-06T23:44Z] 68 were called Somatic [2017-06-06T23:44Z] 2 were called Unknown [2017-06-06T23:44Z] 0 were called Variant [2017-06-06T23:44Z] Varscan paired fix [2017-06-06T23:44Z] Varscan paired fix [2017-06-06T23:44Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-06T23:44Z] Configuring 1 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-06T23:44Z] tabix index syn3-2_62227535_95457199-snp-fix.vcf.gz [2017-06-06T23:44Z] tabix index syn3-2_62227535_95457199-indel-fix.vcf.gz [2017-06-06T23:44Z] Combine variant files [2017-06-06T23:44Z] Exception in thread "main" java.lang.UnsupportedClassVersionError: org/broadinstitute/gatk/engine/CommandLineGATK : Unsupported major.minor version 52.0 [2017-06-06T23:44Z] at java.lang.ClassLoader.defineClass1(Native Method) [2017-06-06T23:44Z] at java.lang.ClassLoader.defineClass(ClassLoader.java:803) [2017-06-06T23:44Z] at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) [2017-06-06T23:44Z] at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) [2017-06-06T23:44Z] at java.net.URLClassLoader.access$100(URLClassLoader.java:64) [2017-06-06T23:44Z] at java.net.URLClassLoader$1.run(URLClassLoader.java:354) [2017-06-06T23:44Z] at java.net.URLClassLoader$1.run(URLClassLoader.java:348) [2017-06-06T23:44Z] at java.security.AccessController.doPrivileged(Native Method) [2017-06-06T23:44Z] at java.net.URLClassLoader.findClass(URLClassLoader.java:347) [2017-06-06T23:44Z] at java.lang.ClassLoader.loadClass(ClassLoader.java:425) [2017-06-06T23:44Z] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) [2017-06-06T23:44Z] at java.lang.ClassLoader.loadClass(ClassLoader.java:358) [2017-06-06T23:44Z] at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) [2017-06-06T23:44Z] Uncaught exception occurred Traceback (most recent call last): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 22, in run _do_run(cmd, checks, log_stdout, env=env) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 102, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && export PATH=/mnt/workspaceXFS/local/share/bcbio/anaconda/bin:$PATH && /mnt/workspaceXFS/local/share/bcbio/anaconda/bin/gatk-framework -Xms750m -Xmx3500m -XX:+UseSerialGC -Djava.io.tmpdir=/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmpfhYWrc -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /mnt/workspaceXFS/local/share/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --out /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmpfhYWrc/syn3-2_62227535_95457199.vcf.gz --variant:v0 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmpMB1T9U/syn3-2_62227535_95457199-snp-fix.vcf.gz --variant:v1 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/bcbiotx/tmpMB1T9U/syn3-2_62227535_95457199-indel-fix.vcf.gz --rod_priority_list v0,v1 --genotypemergeoption PRIORITIZE --suppressCommandLineHeader --setKey null -L /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/work/varscan/2/syn3-2_62227535_95457199-regions-nolcr.bed --interval_set_rule INTERSECTION Exception in thread "main" java.lang.UnsupportedClassVersionError: org/broadinstitute/gatk/engine/CommandLineGATK : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) at java.net.URLClassLoader.access$100(URLClassLoader.java:64) at java.net.URLClassLoader$1.run(URLClassLoader.java:354) at java.net.URLClassLoader$1.run(URLClassLoader.java:348) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:347) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) ' returned non-zero exit status 1 An unexpected error occurred while tokenizing input file /home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/samtools.pyc The following traceback may be corrupted or invalid The error message is: ('EOF in multi-line statement', (12, 0))

[MurliNair]

I guess it is linked to Java version. I am running 1.7 . I do have 1.8, hopefully if I change that it may be okay.

[MurliNair]

I tried with Java version 1.8, it does not run at all. Any thoughts.. Thx ../Murli

[chapmanb]

Murli; Sorry about the Java issues, that is unexpected. You should only need java 1.7 installed externally if you're using MuTect. For all other tools, bcbio ships with java 1.8 and should be using that. There was a fix put in recently for avoiding use of external java if it's in /bin/java, I wonder if it's possible you might not have the latest gatk-framework:

$ bcbio_conda list | grep gatk-framework
gatk-framework            3.6.24                        3    bioconda

If you have an older sub version (not 3), you could update with:

bcbio_conda install -c conda-forge -c bioconda gatk-framework

Hope that explains what is going on. If not, you could try manually debugging by adding echo $java to line 56 of your gatk-framework wrapper script installed with bcbio to figure out what java it picks up during the run. Hope this helps.

[MurliNair]

Brad, The version of java that is enabled system wide is 1.7 (externally) [mnair@bioinformatics ~]$ java -version java version "1.7.0_141" OpenJDK Runtime Environment (rhel-2.6.10.1.el7_3-x86_64 u141-b02) OpenJDK 64-Bit Server VM (build 24.141-b02, mixed mode)

I am running the latest gatk-framework [mnair@bioinformatics ~]$ bcbio_conda list | grep gatk-framework gatk-framework 3.6.24 3 bioconda

Yes, there is a java 1.8 instance in /local/share/bcbio/anaconda/bin [mnair@bioinformatics bin]$ pwd /home/mnair/local/share/bcbio/anaconda/bin [mnair@bioinformatics bin]$ ./java -version openjdk version "1.8.0_121" OpenJDK Runtime Environment (Zulu 8.20.0.5-linux64) (build 1.8.0_121-b15) OpenJDK 64-Bit Server VM (Zulu 8.20.0.5-linux64) (build 25.121-b15, mixed mode)

There is no sub-directory under "/local/share/bcbio/anaconda/bin" named java, but just the executable.

There is also a java 1.8 in local/share/bcbio/anaconda/jre/bin [mnair@bioinformatics bin]$ pwd /home/mnair/local/share/bcbio/anaconda/jre/bin [mnair@bioinformatics bin]$ ./java -version openjdk version "1.8.0_121" OpenJDK Runtime Environment (Zulu 8.20.0.5-linux64) (build 1.8.0_121-b15) OpenJDK 64-Bit Server VM (Zulu 8.20.0.5-linux64) (build 25.121-b15, mixed mode)

Kindly let me know if anything is amiss here.

Is there a way to limit the run to just check if all the installations are correct, before running a long test run. The cancer/normal takes a long time.

[MurliNair]

I ran the test.sh script and got the following . E CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && export PATH=/mnt/workspaceXFS/local/share/bcbio/anaconda/bin:$PATH && /mnt/workspaceXFS/local/share/bcbio/anaconda/bin/gatk-framework -Xms750m -Xmx3500m -XX:+UseSerialGC -Djava.io.tmpdir=/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/bcbiotx/tmpa7Osyb -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa --out /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/bcbiotx/tmpa7Osyb/S1-variants-merge.vcf.gz --variant:v0 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/variants/S1-variants-snp.vcf.gz --variant:v1 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/variants/S1-variants-indel.vcf.gz --rod_priority_list v0,v1 --genotypemergeoption PRIORITIZE --suppressCommandLineHeader --setKey null E Exception in thread "main" java.lang.UnsupportedClassVersionError: org/broadinstitute/gatk/engine/CommandLineGATK : Unsupported major.minor version 52.0 E at java.lang.ClassLoader.defineClass1(Native Method) E at java.lang.ClassLoader.defineClass(ClassLoader.java:803) E at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) E at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) E at java.net.URLClassLoader.access$100(URLClassLoader.java:64) E at java.net.URLClassLoader$1.run(URLClassLoader.java:354) E at java.net.URLClassLoader$1.run(URLClassLoader.java:348) E at java.security.AccessController.doPrivileged(Native Method) E at java.net.URLClassLoader.findClass(URLClassLoader.java:347) E at java.lang.ClassLoader.loadClass(ClassLoader.java:425) E at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) E at java.lang.ClassLoader.loadClass(ClassLoader.java:358) E at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) E ' returned non-zero exit status 1

../bcbio/provenance/do.py:102: CalledProcessError ================================================= 57 tests deselected ================================================== ================================= 3 failed, 8 passed, 57 deselected in 221.92 seconds

[chapmanb]

Murli; Thanks for following up. I'm glad you found the unit tests, as that's the best place to test this. You should be able to do ./run_tests.sh cancerpanel to reproduce since that uses MuTect.

From reading your details above, I'm confused about the installation you're using. You mention:

 /home/mnair/local/share/bcbio/anaconda/bin

but bcbio, from the error tracebacks seems to use:

/mnt/workspaceXFS/local/share/bcbio/anaconda/bin:

are these the same due to symlinks or multiple installations? My best recommendation at this point is to edit the wrapper script:

/mnt/workspaceXFS/local/share/bcbio/anaconda/bin/gatk-framework

and add:

`echo $DIR $java

on line 56 just before all the eval steps. That should at least let us know where it it picking up java from and how it gets your system java instead of the bcbio installed java 8.

Hope this helps isolate the problem.

[MurliNair]

Brad, /home/mnair/local is a symlink to /mnt/workspaceXFS/local Here is the output of ./run_tests.sh cancerpanel after I added echo $DIR $java to gatk-framework .

======================================= test session starts ======================================== platform linux2 -- Python 2.7.13, pytest-3.1.1, py-1.4.33, pluggy-0.4.0 -- /home/mnair/local/share/bcbio/anaconda/bin/python cachedir: ../.cache rootdir: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen, inifile: plugins: cov-2.4.0, marks-0.4, mock-1.1 collected 68 items

integration/test_automated_analysis.py::test_7_cancer_nonormal [2017-06-08T00:37Z] System YAML configuration: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/bcbio_system.yaml [2017-06-08T00:37Z] Resource requests: bwa, sambamba, samtools; memory: 3.00, 3.00, 3.00; cores: 16, 16, 16 [2017-06-08T00:37Z] Configuring 1 jobs to run, using 1 cores each with 3.00g of memory reserved for each job [2017-06-08T00:37Z] Timing: organize samples [2017-06-08T00:37Z] multiprocessing: organize_samples [2017-06-08T00:37Z] Using input YAML configuration: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-cancer2.yaml [2017-06-08T00:37Z] Checking sample YAML configuration: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-cancer2.yaml [2017-06-08T00:37Z] Testing minimum versions of installed programs [2017-06-08T00:37Z] Timing: alignment preparation [2017-06-08T00:37Z] multiprocessing: prep_align_inputs [2017-06-08T00:37Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-08T00:37Z] Configuring 1 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-08T00:37Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-08T00:37Z] Configuring 1 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-08T00:37Z] Index input with grabix: c-tumor_icgc_syn3_tumor.fq.gz [2017-06-08T00:37Z] multiprocessing: disambiguate_split [2017-06-08T00:37Z] Timing: alignment [2017-06-08T00:37Z] multiprocessing: process_alignment [2017-06-08T00:37Z] Aligning lane 1_2014-02-05_tcancer2 with bwa aligner [2017-06-08T00:37Z] bwa mem alignment from fastq: c-tumor [2017-06-08T00:37Z] [M::mem_pestat] skip orientation FF as there are not enough pairs [2017-06-08T00:37Z] [M::mem_pestat] analyzing insert size distribution for orientation FR... [2017-06-08T00:37Z] [M::mem_pestat] (25, 50, 75) percentile: (97, 327, 344) [2017-06-08T00:37Z] [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 838) [2017-06-08T00:37Z] [M::mem_pestat] mean and std.dev: (259.72, 135.86) [2017-06-08T00:37Z] [M::mem_pestat] low and high boundaries for proper pairs: (1, 1085) [2017-06-08T00:37Z] [M::mem_pestat] skip orientation RF as there are not enough pairs [2017-06-08T00:37Z] [M::mem_pestat] skip orientation RR as there are not enough pairs [2017-06-08T00:37Z] [main] Version: 0.7.15-r1140 [2017-06-08T00:37Z] [main] CMD: /home/mnair/local/share/bcbio/anaconda/bin/bwa mem -p -c 250 -M -t 1 -R @RG\tID:c-tumor\tPL:illumina\tPU:1_2014-02-05_tcancer2\tSM:c-tumor -v 1 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/bwa/hg19.fa /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align_prep/c-tumor_icgc_syn3_tumor.fq.gz [2017-06-08T00:37Z] [main] Real time: 0.848 sec; CPU: 0.848 sec [2017-06-08T00:37Z] [V] 12101 89138000 MemUsage(size=469.996,rss=18.957,peak=469.996) AutoArrayMemUsage(memusage=338.961,peakmemusage=338.961,maxmem=1.75922e+13) final [2017-06-08T00:37Z] [V] flushing read ends lists...done. [2017-06-08T00:37Z] [V] merging read ends lists/computing duplicates...done, time 01:00097100 [2017-06-08T00:37Z] [V] num dups 2185 [2017-06-08T00:37Z] # bamsormadup [2017-06-08T00:37Z] [2017-06-08T00:37Z] ##METRICS [2017-06-08T00:37Z] LIBRARY UNPAIRED_READS_EXAMINED READ_PAIRS_EXAMINED UNMAPPED_READS UNPAIRED_READ_DUPLICATES READ_PAIR_DUPLICATES READ_PAIR_OPTICAL_DUPLICATES PERCENT_DUPLICATIONESTIMATED_LIBRARY_SIZE [2017-06-08T00:37Z] Unknown Library 11573 93 339 2177 4 0 0.185815 1049 [2017-06-08T00:37Z] [2017-06-08T00:37Z] ## HISTOGRAM [2017-06-08T00:37Z] BIN VALUE [2017-06-08T00:37Z] 1 0.999963 [2017-06-08T00:37Z] 2 1.91509 [2017-06-08T00:37Z] 3 2.75258 [2017-06-08T00:37Z] 4 3.51901 [2017-06-08T00:37Z] 5 4.22042 [2017-06-08T00:37Z] 6 4.86233 [2017-06-08T00:37Z] 7 5.44977 [2017-06-08T00:37Z] 8 5.98738 [2017-06-08T00:37Z] 9 6.47937 [2017-06-08T00:37Z] 10 6.92963 [2017-06-08T00:37Z] 11 7.34169 [2017-06-08T00:37Z] 12 7.71878 [2017-06-08T00:37Z] 13 8.06389 [2017-06-08T00:37Z] 14 8.37971 [2017-06-08T00:37Z] 15 8.66875 [2017-06-08T00:37Z] 16 8.93326 [2017-06-08T00:37Z] 17 9.17533 [2017-06-08T00:37Z] 18 9.39686 [2017-06-08T00:37Z] 19 9.5996 [2017-06-08T00:37Z] 20 9.78513 [2017-06-08T00:37Z] 21 9.95493 [2017-06-08T00:37Z] 22 10.1103 [2017-06-08T00:37Z] 23 10.2525 [2017-06-08T00:37Z] 24 10.3827 [2017-06-08T00:37Z] 25 10.5018 [2017-06-08T00:37Z] 26 10.6108 [2017-06-08T00:37Z] 27 10.7105 [2017-06-08T00:37Z] 28 10.8018 [2017-06-08T00:37Z] 29 10.8853 [2017-06-08T00:37Z] 30 10.9618 [2017-06-08T00:37Z] 31 11.0318 [2017-06-08T00:37Z] 32 11.0958 [2017-06-08T00:37Z] 33 11.1544 [2017-06-08T00:37Z] 34 11.208 [2017-06-08T00:37Z] 35 11.2571 [2017-06-08T00:37Z] 36 11.302 [2017-06-08T00:37Z] 37 11.3431 [2017-06-08T00:37Z] 38 11.3807 [2017-06-08T00:37Z] 39 11.4152 [2017-06-08T00:37Z] 40 11.4467 [2017-06-08T00:37Z] 41 11.4755 [2017-06-08T00:37Z] 42 11.5019 [2017-06-08T00:37Z] 43 11.526 [2017-06-08T00:37Z] 44 11.5481 [2017-06-08T00:37Z] 45 11.5684 [2017-06-08T00:37Z] 46 11.5869 [2017-06-08T00:37Z] 47 11.6038 [2017-06-08T00:37Z] 48 11.6193 [2017-06-08T00:37Z] 49 11.6335 [2017-06-08T00:37Z] 50 11.6465 [2017-06-08T00:37Z] 51 11.6584 [2017-06-08T00:37Z] 52 11.6692 [2017-06-08T00:37Z] 53 11.6792 [2017-06-08T00:37Z] 54 11.6883 [2017-06-08T00:37Z] 55 11.6966 [2017-06-08T00:37Z] 56 11.7042 [2017-06-08T00:37Z] 57 11.7112 [2017-06-08T00:37Z] 58 11.7176 [2017-06-08T00:37Z] 59 11.7235 [2017-06-08T00:37Z] 60 11.7288 [2017-06-08T00:37Z] 61 11.7337 [2017-06-08T00:37Z] 62 11.7382 [2017-06-08T00:37Z] 63 11.7423 [2017-06-08T00:37Z] 64 11.746 [2017-06-08T00:37Z] 65 11.7495 [2017-06-08T00:37Z] 66 11.7526 [2017-06-08T00:37Z] 67 11.7555 [2017-06-08T00:37Z] 68 11.7581 [2017-06-08T00:37Z] 69 11.7605 [2017-06-08T00:37Z] 70 11.7627 [2017-06-08T00:37Z] 71 11.7648 [2017-06-08T00:37Z] 72 11.7666 [2017-06-08T00:37Z] 73 11.7683 [2017-06-08T00:37Z] 74 11.7698 [2017-06-08T00:37Z] 75 11.7713 [2017-06-08T00:37Z] 76 11.7725 [2017-06-08T00:37Z] 77 11.7737 [2017-06-08T00:37Z] 78 11.7748 [2017-06-08T00:37Z] 79 11.7758 [2017-06-08T00:37Z] 80 11.7767 [2017-06-08T00:37Z] 81 11.7775 [2017-06-08T00:37Z] 82 11.7783 [2017-06-08T00:37Z] 83 11.779 [2017-06-08T00:37Z] 84 11.7796 [2017-06-08T00:37Z] 85 11.7802 [2017-06-08T00:37Z] 86 11.7808 [2017-06-08T00:37Z] 87 11.7812 [2017-06-08T00:37Z] 88 11.7817 [2017-06-08T00:37Z] 89 11.7821 [2017-06-08T00:37Z] 90 11.7825 [2017-06-08T00:37Z] 91 11.7828 [2017-06-08T00:37Z] 92 11.7831 [2017-06-08T00:37Z] 93 11.7834 [2017-06-08T00:37Z] 94 11.7837 [2017-06-08T00:37Z] 95 11.7839 [2017-06-08T00:37Z] 96 11.7841 [2017-06-08T00:37Z] 97 11.7843 [2017-06-08T00:37Z] 98 11.7845 [2017-06-08T00:37Z] 99 11.7847 [2017-06-08T00:37Z] 100 11.7849 [2017-06-08T00:37Z] [V] blocks generated in time 04:00931499 [2017-06-08T00:37Z] [V] number of blocks to be merged is 1 using 8192 blocks per input with block size 1048576 [2017-06-08T00:37Z] [V] 12101 [2017-06-08T00:37Z] [D] md5 6bcc9d9561ef207819ed3a52c740ced0 [2017-06-08T00:37Z] [V] checksum ok [2017-06-08T00:37Z] [V] blocks merged in time 01:07416299 [2017-06-08T00:37Z] [V] run time 05:08414300 (5.08414 s) MemUsage(size=8344.02,rss=70.9648,peak=9368.04) [2017-06-08T00:37Z] Index BAM file with sambamba: c-tumor-sort.bam [2017-06-08T00:37Z] Timing: callable regions [2017-06-08T00:37Z] multiprocessing: prep_samples [2017-06-08T00:37Z] multiprocessing: postprocess_alignment [2017-06-08T00:37Z] Counting not unmapped and not mate_is_unmapped and not secondary_alignment and not failed_quality_control and not duplicate for /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align/c-tumor/c-tumor-sort.bam [2017-06-08T00:37Z] Calculate coverage: c-tumor [2017-06-08T00:37Z] c-tumor: Assigned coverage as 'genome' with 54.7% genome coverage and 0.0% offtarget coverage [2017-06-08T00:37Z] Prepare cleaned BED file : c-tumor [2017-06-08T00:37Z] bgzip c-tumor-coverage.callable-vrsubset-callableblocks.bed [2017-06-08T00:37Z] tabix index c-tumor-coverage.callable-vrsubset-callableblocks.bed.gz [2017-06-08T00:37Z] Prepare merged BED file : c-tumor [2017-06-08T00:37Z] bgzip c-tumor-coverage.callable-vrsubset-callableblocks-merged.bed [2017-06-08T00:37Z] tabix index c-tumor-coverage.callable-vrsubset-callableblocks-merged.bed.gz [2017-06-08T00:37Z] multiprocessing: combine_sample_regions [2017-06-08T00:37Z] Identified 1 parallel analysis blocks Block sizes: [20001] Between block sizes: []

[2017-06-08T00:37Z] Timing: hla typing [2017-06-08T00:37Z] Resource requests: freebayes, gatk, mutect, picard, vardict; memory: 3.00, 3.50, 3.00, 3.00, 3.00; cores: 16, 1, 16, 16, 16 [2017-06-08T00:37Z] Configuring 1 jobs to run, using 1 cores each with 3.50g of memory reserved for each job [2017-06-08T00:37Z] Timing: alignment post-processing [2017-06-08T00:37Z] multiprocessing: piped_bamprep [2017-06-08T00:37Z] Timing: variant calling [2017-06-08T00:37Z] multiprocessing: variantcall_sample Traceback (most recent call last): File "/home/mnair/local/bin/bcbio_nextgen.py", line 234, in main(kwargs) File "/home/mnair/local/bin/bcbio_nextgen.py", line 43, in main run_main(kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 42, in run_main fc_dir, run_info_yaml) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 86, in _run_toplevel for xs in pipeline(config, run_info_yaml, parallel, dirs, samples): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 149, in variant2pipeline samples = genotype.parallel_variantcall_region(samples, run_parallel) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/genotype.py", line 181, in parallel_variantcall_region "vrn_file", ["region", "sam_ref", "config"])) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/split.py", line 32, in grouped_parallel_split_combine final_output = parallel_fn(parallel_name, split_args) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel return run_multicore(fn, items, config, parallel=parallel) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore for data in joblib.Parallel(parallel["num_jobs"], batch_size=1)(joblib.delayed(fn)(x) for x in items): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 804, in call while self.dispatch_one_batch(iterator): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 662, in dispatch_one_batch self._dispatch(tasks) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 570, in _dispatch job = ImmediateComputeBatch(batch) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 183, in init self.results = batch() File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in call return [func(*args, *kwargs) for func, args, kwargs in self.items] File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 51, in wrapper return apply(f, args, *kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 254, in variantcall_sample return genotype.variantcall_sample(args) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/genotype.py", line 317, in variantcall_sample call_file = caller_fn(align_bams, items, sam_ref, assoc_files, region, call_file) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/mutect.py", line 122, in mutect_caller region, out_file_mutect) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/mutect.py", line 86, in _mutect_call_prep _check_mutect_version(broad_runner) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/mutect.py", line 27, in _check_mutect_version mutect_version = broad_runner.get_mutect_version() File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/broad/init.py", line 344, in get_mutect_version mutect_jar = self._get_jar("muTect", ["mutect"]) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/broad/init.py", line 422, in _get_jar raise ValueError("Could not find jar %s in %s:%s" % (command, self._picard_ref, self._gatk_dir)) ValueError: Could not find jar muTect in /home/mnair/local/share/bcbio/anaconda/bin/picard:/home/mnair/local/share/bcbio/anaconda/bin/picard FAILED

============================================= FAILURES ============================================= __ test_7_cancer_nonormal __

install_test_files = None data_dir = '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated'

@pytest.mark.cancer
@pytest.mark.cancerpanel
@pytest.mark.install_required
def test_7_cancer_nonormal(install_test_files, data_dir):
    """Test cancer calling without normal samples or with normal VCF panels.
    Requires MuTect and GATK.
    """
    with make_workdir() as workdir:
        cl = ["bcbio_nextgen.py",
              get_post_process_yaml(data_dir, workdir),
              os.path.join(data_dir, "run_info-cancer2.yaml")]

      subprocess.check_call(cl)

integration/test_automated_analysis.py:255:

---

popenargs = (['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_au...spaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-cancer2.yaml'],) kwargs = {}, retcode = 1 cmd = ['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_aut...rkspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-cancer2.yaml']

def check_call(*popenargs, **kwargs):
    """Run command with arguments.  Wait for command to complete.  If
    the exit code was zero then return, otherwise raise
    CalledProcessError.  The CalledProcessError object will have the
    return code in the returncode attribute.

    The arguments are the same as for the Popen constructor.  Example:

    check_call(["ls", "-l"])
    """
    retcode = call(*popenargs, **kwargs)
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]

      raise CalledProcessError(retcode, cmd)

E CalledProcessError: Command '['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/bcbio_system.yaml', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-cancer2.yaml']' returned non-zero exit status 1

/home/mnair/local/share/bcbio/anaconda/lib/python2.7/subprocess.py:186: CalledProcessError ======================================= 67 tests deselected ======================================== ============================= 1 failed, 67 deselected in 17.37 seconds ===========================

Again, I really appreciate you taking the time to help me with the installation.

[chapmanb]

Murli; Apologies, I feel like I've thrown you off with the test I asked to run since you didn't install MuTect as part of the installation. Could you try running ./run_tests.sh devel to see if that replicates the java gatk-framework error so we can determine what the java resolution issue is? Sorry to introduce a different unrelated error and hopefully this gets us back on track.

[MurliNair]

Brad, Not a problem, happy to run any script. Here is the output, I was not able to capture everything even after redirecting it to a file.

[2017-06-08T13:00Z] INFO 09:00:43,642 StrandBiasTest - SAM/BAM data was found. Attempting to use read data to calculate strand bias annotations values. [2017-06-08T13:00Z] WARN 09:00:43,751 HaplotypeScore - Annotation will not be calculated, must be called from UnifiedGenotyper, not org.broadinstitute.gatk.tools.walkers.annotator.VariantAnnotator [2017-06-08T13:00Z] INFO 09:00:43,774 VariantAnnotator - Processed 4 loci. [2017-06-08T13:00Z] [2017-06-08T13:00Z] INFO 09:00:43,780 ProgressMeter - done 18.0 0.0 s 3.0 h 83.3% 0.0 s 0.0 s [2017-06-08T13:00Z] INFO 09:00:43,781 ProgressMeter - Total runtime 0.19 secs, 0.00 min, 0.00 hours [2017-06-08T13:00Z] INFO 09:00:43,781 MicroScheduler - 0 reads were filtered out during the traversal out of approximately 9 total reads (0.00%) [2017-06-08T13:00Z] INFO 09:00:43,781 MicroScheduler - -> 0 reads (0.00% of total) failing BadCigarFilter [2017-06-08T13:00Z] INFO 09:00:43,782 MicroScheduler - -> 0 reads (0.00% of total) failing DuplicateReadFilter [2017-06-08T13:00Z] INFO 09:00:43,782 MicroScheduler - -> 0 reads (0.00% of total) failing FailsVendorQualityCheckFilter [2017-06-08T13:00Z] INFO 09:00:43,782 MicroScheduler - -> 0 reads (0.00% of total) failing MalformedReadFilter [2017-06-08T13:00Z] INFO 09:00:43,782 MicroScheduler - -> 0 reads (0.00% of total) failing NotPrimaryAlignmentFilter [2017-06-08T13:00Z] INFO 09:00:43,782 MicroScheduler - -> 0 reads (0.00% of total) failing UnmappedReadFilter [2017-06-08T13:00Z] ------------------------------------------------------------------------------------------ [2017-06-08T13:00Z] Done. There were 3 WARN messages, the first 3 are repeated below. [2017-06-08T13:00Z] WARN 09:00:43,500 IndexDictionaryUtils - Track variant doesn't have a sequence dictionary built in, skipping dictionary validation [2017-06-08T13:00Z] WARN 09:00:43,501 IndexDictionaryUtils - Track dbsnp doesn't have a sequence dictionary built in, skipping dictionary validation [2017-06-08T13:00Z] WARN 09:00:43,751 HaplotypeScore - Annotation will not be calculated, must be called from UnifiedGenotyper, not org.broadinstitute.gatk.tools.walkers.annotator.VariantAnnotator [2017-06-08T13:00Z] ------------------------------------------------------------------------------------------ [2017-06-08T13:00Z] Genotyping with FreeBayes [2017-06-08T13:00Z] tabix index TestBatch1-chr22_15128_15500.vcf.gz [2017-06-08T13:00Z] GATK: VariantAnnotator [2017-06-08T13:00Z] INFO 09:00:51,487 HelpFormatter - ---------------------------------------------------------------------------------- [2017-06-08T13:00Z] INFO 09:00:51,490 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.7-0-gcfedb67, Compiled 2016/12/12 11:21:18 [2017-06-08T13:00Z] INFO 09:00:51,490 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute [2017-06-08T13:00Z] INFO 09:00:51,490 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk [2017-06-08T13:00Z] INFO 09:00:51,490 HelpFormatter - [Thu Jun 08 09:00:51 EDT 2017] Executing on Linux 3.10.0-514.16.1.el7.x86_64 amd64 [2017-06-08T13:00Z] INFO 09:00:51,490 HelpFormatter - OpenJDK 64-Bit Server VM 1.8.0_121-b15 [2017-06-08T13:00Z] INFO 09:00:51,494 HelpFormatter - Program Args: -T VariantAnnotator -R /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa --variant /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/freebayes/chr22/TestBatch1-chr22_15128_15500.vcf.gz --out /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmpUS1leu/TestBatch1-chr22_15128_15500-gatkann.vcf.gz -L /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/freebayes/chr22/TestBatch1-chr22_15128_15500.vcf.gz --dbsnp /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/variation/dbsnp_132.vcf.gz -I /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align/Test1/Test1-sort.bam -A BaseQualityRankSumTest -A FisherStrand -A GCContent -A HaplotypeScore -A HomopolymerRun -A MappingQualityRankSumTest -A MappingQualityZero -A QualByDepth -A ReadPosRankSumTest -A RMSMappingQuality --allow_potentially_misencoded_quality_scores -U ALL --read_filter BadCigar --read_filter NotPrimaryAlignment [2017-06-08T13:00Z] INFO 09:00:51,498 HelpFormatter - Executing as mnair@bioinformatics.iusb.edu on Linux 3.10.0-514.16.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_121-b15. [2017-06-08T13:00Z] INFO 09:00:51,498 HelpFormatter - Date/Time: 2017/06/08 09:00:51 [2017-06-08T13:00Z] INFO 09:00:51,498 HelpFormatter - ---------------------------------------------------------------------------------- [2017-06-08T13:00Z] INFO 09:00:51,498 HelpFormatter - ---------------------------------------------------------------------------------- [2017-06-08T13:00Z] INFO 09:00:51,537 GenomeAnalysisEngine - Strictness is SILENT [2017-06-08T13:00Z] INFO 09:00:51,618 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 250 [2017-06-08T13:00Z] INFO 09:00:51,625 SAMDataSource$SAMReaders - Initializing SAMRecords in serial [2017-06-08T13:00Z] INFO 09:00:51,644 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.02 [2017-06-08T13:00Z] WARN 09:00:51,685 IntervalUtils - The interval file /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/freebayes/chr22/TestBatch1-chr22_15128_15500.vcf.gz contains no intervals that could be parsed. [2017-06-08T13:00Z] INFO 09:00:51,686 IntervalUtils - Processing 0 bp from intervals [2017-06-08T13:00Z] WARN 09:00:51,687 GenomeAnalysisEngine - The given combination of -L and -XL options results in an empty set. No intervals to process. [2017-06-08T13:00Z] WARN 09:00:51,690 IndexDictionaryUtils - Track variant doesn't have a sequence dictionary built in, skipping dictionary validation [2017-06-08T13:00Z] WARN 09:00:51,690 IndexDictionaryUtils - Track dbsnp doesn't have a sequence dictionary built in, skipping dictionary validation [2017-06-08T13:00Z] INFO 09:00:51,754 GenomeAnalysisEngine - Preparing for traversal over 1 BAM files [2017-06-08T13:00Z] INFO 09:00:51,756 GenomeAnalysisEngine - Done preparing for traversal [2017-06-08T13:00Z] INFO 09:00:51,757 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING] [2017-06-08T13:00Z] INFO 09:00:51,757 ProgressMeter - | processed | time | per 1M | | total | remaining [2017-06-08T13:00Z] INFO 09:00:51,757 ProgressMeter - Location | sites | elapsed | sites | completed | runtime | runtime [2017-06-08T13:00Z] INFO 09:00:51,813 StrandBiasTest - SAM/BAM data was found. Attempting to use read data to calculate strand bias annotations values. [2017-06-08T13:00Z] INFO 09:00:51,843 VariantAnnotator - Processed 0 loci. [2017-06-08T13:00Z] [2017-06-08T13:00Z] INFO 09:00:51,848 ProgressMeter - done 0.0 0.0 s 25.0 h 0.0% 0.0 s 0.0 s [2017-06-08T13:00Z] INFO 09:00:51,848 ProgressMeter - Total runtime 0.09 secs, 0.00 min, 0.00 hours [2017-06-08T13:00Z] ------------------------------------------------------------------------------------------ [2017-06-08T13:00Z] Done. There were 4 WARN messages, the first 4 are repeated below. [2017-06-08T13:00Z] WARN 09:00:51,685 IntervalUtils - The interval file /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/freebayes/chr22/TestBatch1-chr22_15128_15500.vcf.gz contains no intervals that could be parsed. [2017-06-08T13:00Z] WARN 09:00:51,687 GenomeAnalysisEngine - The given combination of -L and -XL options results in an empty set. No intervals to process. [2017-06-08T13:00Z] WARN 09:00:51,690 IndexDictionaryUtils - Track variant doesn't have a sequence dictionary built in, skipping dictionary validation [2017-06-08T13:00Z] WARN 09:00:51,690 IndexDictionaryUtils - Track dbsnp doesn't have a sequence dictionary built in, skipping dictionary validation [2017-06-08T13:00Z] ------------------------------------------------------------------------------------------ [2017-06-08T13:00Z] multiprocessing: concat_variant_files [2017-06-08T13:00Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-08T13:00Z] Configuring 1 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-08T13:00Z] bcftools concat variants [2017-06-08T13:00Z] tabix index TestBatch1.vcf.gz [2017-06-08T13:00Z] Resource requests: bcbio_variation, cnvkit, delly, fastqc, gatk, gemini, gridss, lumpy, qsignature, sambamba, samtools, seq2c, snpeff; memory: 3.00, 3.00, 3.00, 3.00, 3.50, 3.00, 3.00, 3.00, 3.00, 3.00, 3.00, 3.00, 3.00; cores: 16, 16, 16, 16, 1, 16, 16, 16, 16, 16, 16, 16, 1 [2017-06-08T13:00Z] Configuring 1 jobs to run, using 1 cores each with 3.50g of memory reserved for each job [2017-06-08T13:00Z] Timing: joint squaring off/backfilling [2017-06-08T13:00Z] Timing: variant post-processing [2017-06-08T13:00Z] multiprocessing: postprocess_variants [2017-06-08T13:00Z] Finalizing variant calls: VarOnly, None [2017-06-08T13:00Z] Finalizing variant calls: Test1, freebayes [2017-06-08T13:00Z] Calculating variation effects for Test1, freebayes [2017-06-08T13:00Z] snpEff effects : Test1 [2017-06-08T13:00Z] tabix index TestBatch1-effects.vcf.gz [2017-06-08T13:00Z] Annotate VCF file: Test1, freebayes [2017-06-08T13:00Z] Filtering for Test1, freebayes [2017-06-08T13:00Z] Cutoff-based soft filtering /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/freebayes/TestBatch1-effects.vcf.gz with (AF[0] <= 0.5 && (DP < 4 || (DP < 13 && %QUAL < 10))) || (AF[0] > 0.5 && (DP < 4 && %QUAL < 50)) : Test1 [2017-06-08T13:00Z] tabix index TestBatch1-effects-filter.vcf.gz [2017-06-08T13:00Z] Prioritization for Test1, freebayes [2017-06-08T13:00Z] Germline extraction for Test1, freebayes [2017-06-08T13:00Z] multiprocessing: split_variants_by_sample [2017-06-08T13:00Z] Timing: prepped BAM merging [2017-06-08T13:00Z] Timing: validation [2017-06-08T13:00Z] multiprocessing: compare_to_rm [2017-06-08T13:00Z] bgzip 7_100326_FC6107FAAXX-grade.vcf [2017-06-08T13:00Z] tabix index 7_100326_FC6107FAAXX-grade.vcf.gz [2017-06-08T13:00Z] Counting not unmapped and not mate_is_unmapped and not secondary_alignment and not failed_quality_control and not duplicate for /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align/Test1/Test1-sort.bam [2017-06-08T13:00Z] Calculate coverage: VarOnly Traceback (most recent call last): File "/home/mnair/local/bin/bcbio_nextgen.py", line 234, in main(kwargs) File "/home/mnair/local/bin/bcbio_nextgen.py", line 43, in main run_main(kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 42, in run_main fc_dir, run_info_yaml) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 86, in _run_toplevel for xs in pipeline(config, run_info_yaml, parallel, dirs, samples): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 166, in variant2pipeline samples = run_parallel("compare_to_rm", samples) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel return run_multicore(fn, items, config, parallel=parallel) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore for data in joblib.Parallel(parallel["num_jobs"], batch_size=1)(joblib.delayed(fn)(x) for x in items): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 804, in call while self.dispatch_one_batch(iterator): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 662, in dispatch_one_batch self._dispatch(tasks) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 570, in _dispatch job = ImmediateComputeBatch(batch) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 183, in init self.results = batch() File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in call return [func(*args, *kwargs) for func, args, kwargs in self.items] File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 51, in wrapper return apply(f, args, *kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 306, in compare_to_rm return validate.compare_to_rm(args) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/validate.py", line 150, in compare_to_rm eval_files = _run_rtg_eval(vrn_file, rm_file, rm_interval_file, base_dir, toval_data) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/validate.py", line 253, in _run_rtg_eval cmd += ["--vcf-score-field='%s'" % (_pick_best_quality_score(vrn_file))] File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/validate.py", line 294, in _pick_best_quality_score if rec.info.get("VQSLOD") is not None: File "pysam/libcbcf.pyx", line 2642, in pysam.libcbcf.VariantRecordInfo.get (pysam/libcbcf.c:40036) ValueError: Invalid header FAILED

============================================== FAILURES ============================================== _ test_5bam

install_test_files = None data_dir = '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated'

@pytest.mark.devel
@pytest.mark.speed1
def test_5_bam(install_test_files, data_dir):
    """Allow BAM files as input to pipeline.
    """
    with make_workdir() as workdir:
        cl = ["bcbio_nextgen.py",
              get_post_process_yaml(data_dir, workdir),
              os.path.join(data_dir, "run_info-bam.yaml")]

      subprocess.check_call(cl)

integration/test_automated_analysis.py:215:

---

popenargs = (['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_au...workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-bam.yaml'],) kwargs = {}, retcode = 1 cmd = ['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_aut...t/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-bam.yaml']

def check_call(*popenargs, **kwargs):
    """Run command with arguments.  Wait for command to complete.  If
    the exit code was zero then return, otherwise raise
    CalledProcessError.  The CalledProcessError object will have the
    return code in the returncode attribute.

    The arguments are the same as for the Popen constructor.  Example:

    check_call(["ls", "-l"])
    """
    retcode = call(*popenargs, **kwargs)
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]

      raise CalledProcessError(retcode, cmd)

E CalledProcessError: Command '['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/bcbio_system.yaml', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-bam.yaml']' returned non-zero exit status 1

/home/mnair/local/share/bcbio/anaconda/lib/python2.7/subprocess.py:186: CalledProcessError ======================================== 67 tests deselected ========================================= ============================== 1 failed, 67 deselected in 73.03 seconds ==============================

[chapmanb]

Murli; Sorry again about the extraneous issues. I'm trying to get you a test that replicates the gatk-framework java error you've seen. In your latest report you're hitting an issue with the latest pysam that is fixed in the bcbio development version. I'd recommend updating with:

bcbio_nextgen.py upgrade -u development

Hopefully this test will then pass. It would be helpful if you can identify a test case that fails with the Java error you identified earlier and we can work to debug from there. Hope this helps.

[MurliNair]

Brad, I ran the upgrade which went smoothly. Then I ran ./run_tests devel and it still failed. The following is the output.

[2017-06-08T14:10Z] Sort VCF by reference [2017-06-08T14:10Z] tabix index Test1-sort-TestBatch1-svs-prep-Test1-std-wgts-prep.vcf.gz [2017-06-08T14:10Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-08T14:10Z] Configuring 1 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-08T14:10Z] bcftools concat variants [2017-06-08T14:10Z] tabix index Test1-sort-TestBatch1-svs-prep-Test1-std-wgts-prep-combined.vcf.gz [2017-06-08T14:10Z] tabix index Test1-sort-TestBatch1-svs-prep-Test1-std-wgts-prep-combined-filter.vcf.gz [2017-06-08T14:10Z] Resource requests: delly; memory: 1.00; cores: 1 [2017-06-08T14:10Z] Configuring 1 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-08T14:10Z] delly structural variant [2017-06-08T14:10Z] [2017-Jun-08 10:10:07] delly call -n -t DEL -g /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmp4FT9BF/Test1-sort-TestBatch1-svs-chrM-DEL.bcf -x /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chrM-DEL-exclude-chrM-delly.bed /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align/Test1/Test1-sort.bam [2017-06-08T14:10Z] [2017-Jun-08 10:10:07] Paired-end clustering [2017-06-08T14:10Z] 0% 10 20 30 40 50 60 70 80 90 100% [2017-06-08T14:10Z] |----|----|----|----|----|----|----|----|----|----| [2017-06-08T14:10Z] [2017-06-08T14:10Z] [2017-Jun-08 10:10:07] Library statistics [2017-06-08T14:10Z] Sample: Test1 [2017-06-08T14:10Z] RG: ID=Test1,ReadSize=76,AvgDist=0,EstCov=inf,MappedAsPair=1,Median=151,MAD=23,Layout=2,MaxSize=358,MinSize=0,UniqueDiscordantPairs=0 [2017-06-08T14:10Z] [2017-Jun-08 10:10:07] Done. [2017-06-08T14:10Z] Convert and clean delly output [2017-06-08T14:10Z] tabix index Test1-sort-TestBatch1-svs-chrM-DEL.vcf.gz [2017-06-08T14:10Z] delly structural variant [2017-06-08T14:10Z] [2017-Jun-08 10:10:07] delly call -n -t DUP -g /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmpjr83an/Test1-sort-TestBatch1-svs-chrM-DUP.bcf -x /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chrM-DUP-exclude-chrM-delly.bed /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align/Test1/Test1-sort.bam [2017-06-08T14:10Z] [2017-Jun-08 10:10:07] Paired-end clustering [2017-06-08T14:10Z] 0% 10 20 30 40 50 60 70 80 90 100% [2017-06-08T14:10Z] |----|----|----|----|----|----|----|----|----|----| [2017-06-08T14:10Z] [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] Library statistics [2017-06-08T14:10Z] Sample: Test1 [2017-06-08T14:10Z] RG: ID=Test1,ReadSize=76,AvgDist=0,EstCov=inf,MappedAsPair=1,Median=151,MAD=23,Layout=2,MaxSize=358,MinSize=0,UniqueDiscordantPairs=0 [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] Done. [2017-06-08T14:10Z] Convert and clean delly output [2017-06-08T14:10Z] tabix index Test1-sort-TestBatch1-svs-chrM-DUP.vcf.gz [2017-06-08T14:10Z] delly structural variant [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] delly call -n -t DEL -g /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmp0C3Ukp/Test1-sort-TestBatch1-svs-chr22-DEL.bcf -x /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chr22-DEL-exclude-chr22-delly.bed /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align/Test1/Test1-sort.bam [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] Paired-end clustering [2017-06-08T14:10Z] 0% 10 20 30 40 50 60 70 80 90 100% [2017-06-08T14:10Z] |----|----|----|----|----|----|----|----|----|----| [2017-06-08T14:10Z] [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] Library statistics [2017-06-08T14:10Z] Sample: Test1 [2017-06-08T14:10Z] RG: ID=Test1,ReadSize=100,AvgDist=65,EstCov=1.53846,MappedAsPair=0.992,Median=0,MAD=0,Layout=0,MaxSize=0,MinSize=0,UniqueDiscordantPairs=0 [2017-06-08T14:10Z] Convert and clean delly output [2017-06-08T14:10Z] tabix index Test1-sort-TestBatch1-svs-chr22-DEL.vcf.gz [2017-06-08T14:10Z] delly structural variant [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] delly call -n -t DUP -g /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa -o /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmp7s9irw/Test1-sort-TestBatch1-svs-chr22-DUP.bcf -x /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chr22-DUP-exclude-chr22-delly.bed /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/align/Test1/Test1-sort.bam [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] Paired-end clustering [2017-06-08T14:10Z] 0% 10 20 30 40 50 60 70 80 90 100% [2017-06-08T14:10Z] |----|----|----|----|----|----|----|----|----|----| [2017-06-08T14:10Z] [2017-06-08T14:10Z] [2017-Jun-08 10:10:08] Library statistics [2017-06-08T14:10Z] Sample: Test1 [2017-06-08T14:10Z] RG: ID=Test1,ReadSize=100,AvgDist=65,EstCov=1.53846,MappedAsPair=0.992,Median=0,MAD=0,Layout=0,MaxSize=0,MinSize=0,UniqueDiscordantPairs=0 [2017-06-08T14:10Z] Convert and clean delly output [2017-06-08T14:10Z] tabix index Test1-sort-TestBatch1-svs-chr22-DUP.vcf.gz [2017-06-08T14:10Z] Resource requests: ; memory: 1.00; cores: 1 [2017-06-08T14:10Z] Configuring 1 jobs to run, using 1 cores each with 1.00g of memory reserved for each job [2017-06-08T14:10Z] Combine variant files [2017-06-08T14:10Z] /mnt/workspaceXFS/local/share/bcbio/anaconda/share/gatk-framework-3.6.24-3 /bin/java [2017-06-08T14:10Z] Exception in thread "main" java.lang.UnsupportedClassVersionError: org/broadinstitute/gatk/engine/CommandLineGATK : Unsupported major.minor version 52.0 [2017-06-08T14:10Z] at java.lang.ClassLoader.defineClass1(Native Method) [2017-06-08T14:10Z] at java.lang.ClassLoader.defineClass(ClassLoader.java:803) [2017-06-08T14:10Z] at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) [2017-06-08T14:10Z] at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) [2017-06-08T14:10Z] at java.net.URLClassLoader.access$100(URLClassLoader.java:64) [2017-06-08T14:10Z] at java.net.URLClassLoader$1.run(URLClassLoader.java:354) [2017-06-08T14:10Z] at java.net.URLClassLoader$1.run(URLClassLoader.java:348) [2017-06-08T14:10Z] at java.security.AccessController.doPrivileged(Native Method) [2017-06-08T14:10Z] at java.net.URLClassLoader.findClass(URLClassLoader.java:347) [2017-06-08T14:10Z] at java.lang.ClassLoader.loadClass(ClassLoader.java:425) [2017-06-08T14:10Z] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) [2017-06-08T14:10Z] at java.lang.ClassLoader.loadClass(ClassLoader.java:358) [2017-06-08T14:10Z] at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) [2017-06-08T14:10Z] Uncaught exception occurred Traceback (most recent call last): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 22, in run _do_run(cmd, checks, log_stdout, env=env) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 102, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && export PATH=/mnt/workspaceXFS/local/share/bcbio/anaconda/bin:$PATH && /mnt/workspaceXFS/local/share/bcbio/anaconda/bin/gatk-framework -Xms750m -Xmx3500m -XX:+UseSerialGC -Djava.io.tmpdir=/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmpHwPBXT -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa --out /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmpHwPBXT/Test1-sort-TestBatch1-svs.vcf.gz --variant:v0 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chrM-DEL.vcf.gz --variant:v1 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chrM-DUP.vcf.gz --variant:v2 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chr22-DEL.vcf.gz --variant:v3 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chr22-DUP.vcf.gz --rod_priority_list v0,v1,v2,v3 --genotypemergeoption PRIORITIZE --suppressCommandLineHeader --setKey null /mnt/workspaceXFS/local/share/bcbio/anaconda/share/gatk-framework-3.6.24-3 /bin/java Exception in thread "main" java.lang.UnsupportedClassVersionError: org/broadinstitute/gatk/engine/CommandLineGATK : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) at java.net.URLClassLoader.access$100(URLClassLoader.java:64) at java.net.URLClassLoader$1.run(URLClassLoader.java:354) at java.net.URLClassLoader$1.run(URLClassLoader.java:348) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:347) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) ' returned non-zero exit status 1 Traceback (most recent call last): File "/home/mnair/local/bin/bcbio_nextgen.py", line 234, in main(kwargs) File "/home/mnair/local/bin/bcbio_nextgen.py", line 43, in main run_main(kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 42, in run_main fc_dir, run_info_yaml) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 86, in _run_toplevel for xs in pipeline(config, run_info_yaml, parallel, dirs, samples): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 177, in variant2pipeline samples = structural.run(samples, run_parallel, "standard") File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/init.py", line 172, in run for xs in to_process.values())) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel return run_multicore(fn, items, config, parallel=parallel) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore for data in joblib.Parallel(parallel["num_jobs"], batch_size=1)(joblib.delayed(fn)(x) for x in items): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 804, in call while self.dispatch_one_batch(iterator): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 662, in dispatch_one_batch self._dispatch(tasks) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 570, in _dispatch job = ImmediateComputeBatch(batch) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 183, in init self.results = batch() File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 72, in call return [func(*args, *kwargs) for func, args, kwargs in self.items] File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 50, in wrapper return apply(f, args, *kwargs) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 278, in detect_sv return structural.detect_sv(args) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/init.py", line 191, in detect_sv for svdata in caller_fn(items): File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/structural/delly.py", line 173, in run combo_vcf = vcfutils.combine_variant_files(bytype_vcfs, out_file, ref_file, config) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/vcfutils.py", line 445, in combine_variant_files do.run(broad.gatk_cmd("gatk-framework", jvm_opts, params), "Combine variant files") File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 22, in run _do_run(cmd, checks, log_stdout, env=env) File "/home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 102, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) subprocess.CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && export PATH=/mnt/workspaceXFS/local/share/bcbio/anaconda/bin:$PATH && /mnt/workspaceXFS/local/share/bcbio/anaconda/bin/gatk-framework -Xms750m -Xmx3500m -XX:+UseSerialGC -Djava.io.tmpdir=/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmpHwPBXT -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/genomes/hg19/seq/hg19.fa --out /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/tmptest/tmpHwPBXT/Test1-sort-TestBatch1-svs.vcf.gz --variant:v0 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chrM-DEL.vcf.gz --variant:v1 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chrM-DUP.vcf.gz --variant:v2 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chr22-DEL.vcf.gz --variant:v3 /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/structural/Test1/delly/Test1-sort-TestBatch1-svs-chr22-DUP.vcf.gz --rod_priority_list v0,v1,v2,v3 --genotypemergeoption PRIORITIZE --suppressCommandLineHeader --setKey null /mnt/workspaceXFS/local/share/bcbio/anaconda/share/gatk-framework-3.6.24-3 /bin/java Exception in thread "main" java.lang.UnsupportedClassVersionError: org/broadinstitute/gatk/engine/CommandLineGATK : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:442) at java.net.URLClassLoader.access$100(URLClassLoader.java:64) at java.net.URLClassLoader$1.run(URLClassLoader.java:354) at java.net.URLClassLoader$1.run(URLClassLoader.java:348) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:347) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) ' returned non-zero exit status 1 FAILED

============================================================ FAILURES ============================================================= ___ test_5_bam ____

install_test_files = None data_dir = '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated'

@pytest.mark.devel
@pytest.mark.speed1
def test_5_bam(install_test_files, data_dir):
    """Allow BAM files as input to pipeline.
    """
    with make_workdir() as workdir:
        cl = ["bcbio_nextgen.py",
              get_post_process_yaml(data_dir, workdir),
              os.path.join(data_dir, "run_info-bam.yaml")]

      subprocess.check_call(cl)

integration/test_automated_analysis.py:215:

---

popenargs = (['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_au...workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-bam.yaml'],) kwargs = {}, retcode = 1 cmd = ['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_aut...t/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-bam.yaml']

def check_call(*popenargs, **kwargs):
    """Run command with arguments.  Wait for command to complete.  If
    the exit code was zero then return, otherwise raise
    CalledProcessError.  The CalledProcessError object will have the
    return code in the returncode attribute.

    The arguments are the same as for the Popen constructor.  Example:

    check_call(["ls", "-l"])
    """
    retcode = call(*popenargs, **kwargs)
    if retcode:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = popenargs[0]

      raise CalledProcessError(retcode, cmd)

E CalledProcessError: Command '['bcbio_nextgen.py', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/bcbio_system.yaml', '/mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/data/automated/run_info-bam.yaml']' returned non-zero exit status 1

/home/mnair/local/share/bcbio/anaconda/lib/python2.7/subprocess.py:186: CalledProcessError ======================================================= 67 tests deselected ======================================================= ============================================ 1 failed, 67 deselected in 139.84 seconds ===========================================

[MurliNair]

Brad, Do I need to update the shell wrappers you mentioned? Kindly let me know. Thanks

[chapmanb]

Murli; Sorry about the delay, I was waiting to get the fixed packages fully built and they should be in place now if you do:

bcbio_nextgen.py upgrade --tools

Apologies, I thought I'd fixed these bash scripts earlier to avoid calling java present in /bin/bash but had messed up the checking for empty variables. If you update to the latest they should now use the correct java and avoid the problem you're running into. Hope this fixes everything and gets pipelines working cleanly for you. Thank you for the patience and help debugging.

[MurliNair]

Brad, Thanks a lot. I tested it and everything passed when I ran ./run_tests . I shall now test it with the cancer/normal synthetic data set. The following is the output of the test run I greatly appreciate you help in fixing the problem and I am happy I was able to help in a very small way in catching the bugs. Shall keep you posted on cancer/normal run. Cheers../Murli

PASSED integration/test_automated_analysis.py::test_8_template Configuration file created at: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/100326/config/100326.yaml Edit to finalize and run with: cd /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbio-nextgen/tests/test_automated_output/100326/work bcbio_nextgen.py ../config/100326.yaml PASSED integration/test_pipeline.py::TestRunInfo::test_programs [{'program': 'bamtools', 'version': '2.4.0'}, {'program': 'bcbio-nextgen', 'version': ''}, {'program': 'bcbio-variation', 'version': '0.2.6'}, {'program': 'bcftools', 'version': '1.4.1'}, {'program': 'bedtools', 'version': '2.26.0'}, {'program': 'biobambam', 'version': '2.0.72'}, {'program': 'bioconductor-bubbletree', 'version': '2.1.5'}, {'program': 'bowtie2', 'version': '2.2.8'}, {'program': 'bwa', 'version': '0.7.15'}, {'program': 'chanjo', 'version': ''}, {'program': 'cnvkit', 'version': '0.8.6a0'}, {'program': 'cufflinks', 'version': '2.2.1'}, {'program': 'cutadapt', 'version': '1.13'}, {'program': 'fastqc', 'version': '0.11.5'}, {'program': 'featurecounts', 'version': '1.4.4'}, {'program': 'freebayes', 'version': '1.1.0'}, {'program': 'gatk', 'version': '3.7'}, {'program': 'gatk-framework', 'version': '3.6.24'}, {'program': 'gatk4', 'version': '4.0a1.2.7.2'}, {'program': 'gemini', 'version': '0.20.0'}, {'program': 'grabix', 'version': '0.1.8'}, {'program': 'hisat2', 'version': '2.0.5'}, {'program': 'htseq', 'version': '0.7.2'}, {'program': 'lumpy-sv', 'version': '0.2.13'}, {'program': 'manta', 'version': '1.1.0'}, {'program': 'metasv', 'version': '0.4.0'}, {'program': 'mirdeep2', 'version': '2.0.0.7'}, {'program': 'mutect', 'version': '1.1.5'}, {'program': 'novoalign', 'version': '3.07.00'}, {'program': 'novosort', 'version': 'V3.00.02'}, {'program': 'oncofuse', 'version': '1.1.0'}, {'program': 'phylowgs', 'version': '20150714'}, {'program': 'picard', 'version': '2.9.2'}, {'program': 'platypus-variant', 'version': '0.8.1'}, {'program': 'qualimap', 'version': '2.2.2a'}, {'program': 'rna-star', 'version': ''}, {'program': 'rtg-tools', 'version': '3.7.1'}, {'program': 'sailfish', 'version': '0.10.1'}, {'program': 'salmon', 'version': '0.8.2'}, {'program': 'sambamba', 'version': '0.6.6'}, {'program': 'samblaster', 'version': '0.1.24'}, {'program': 'samtools', 'version': '1.4.1'}, {'program': 'scalpel', 'version': '0.5.3'}, {'program': 'seqbuster', 'version': '3.1'}, {'program': 'snpeff', 'version': '4.3i'}, {'program': 'vardict', 'version': '2017.04.18'}, {'program': 'vardict-java', 'version': '1.5.0'}, {'program': 'variant-effect-predictor', 'version': '87'}, {'program': 'varscan', 'version': '2.4.2'}, {'program': 'vcflib', 'version': '1.0.0_rc1'}, {'program': 'vt', 'version': '2015.11.10'}, {'program': 'wham', 'version': '1.7.0.307'}] PASSED integration/test_pipeline.py::TestVCFUtil::test_1_parallel_vcf_combine PASSED integration/test_pipeline.py::TestVCFUtil::test_2_vcf_exclusion PASSED integration/test_pipeline.py::TestVCFUtil::test_3_vcf_split_combine PASSED integration/test_pipeline.py::TestVCFUtil::test_4_vcf_sample_select PASSED integration/test_pipeline.py::TestVCFUtil::test_5_find_fastq_pairs PASSED

======================================================= 57 tests deselected ======================================================= =========================================== 11 passed, 57 deselected in 313.23 seconds

[MurliNair]

Brad, I ran the cancer/normal example. It complained about not finding mutect.jar. I had installed the latest version gatk, which I understand included in the latest version. I have all the output from nohup , but I am appending the last few lines that shows the error. ........................................................................... /home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/mutect.pyc in _check_mutect_version(broad_runner=) 22 "Comparison method violates its general contract!", 23 "java.lang.IllegalArgumentException: " 24 "Comparison method violates its general contract!"]) 25 26 def _check_mutect_version(broad_runner): ---> 27 mutect_version = broad_runner.get_mutect_version() 28 try: 29 assert mutect_version is not None 30 except AssertionError: 31 logger.warn("WARNING")

........................................................................... /home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/broad/init.pyc in get_mutect_version(self=) 354 355 def get_mutect_version(self): 356 """Retrieve the Mutect version. 357 """ 358 if self._mutect_version is None: --> 359 mutect_jar = self._get_jar("muTect", ["mutect"]) 360 self._mutect_version = get_mutect_version(mutect_jar) 361 return self._mutect_version 362 363 def gatk_type(self):

........................................................................... /home/mnair/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/broad/init.pyc in _get_jar(self=, command='muTect', alts=['mutect'], allow_missing=False) 434 else: 435 pass 436 if allow_missing: 437 return None 438 else: --> 439 raise ValueError("Could not find jar %s in %s:%s" % (command, self._picard_ref, self._gatk_dir)) 440 441 def _get_picard_ref(config): 442 """Handle retrieval of Picard for running, handling multiple cases: 443

ValueError: Could not find jar muTect in /home/mnair/local/share/bcbio/anaconda/bin/picard:/home/mnair/local/share/bcbio/anaconda/bin/picard

[chapmanb]

Murli; Sorry for the confusion, MuTect is a different caller than MuTect2 and is not included with GATK. You need to download and install the MuTect jar separately with:

bcbio_nextgen.py upgrade --tools --toolplus mutect=/path/to/mutect/mutect-1.1.7.jar

http://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html#gatk-and-mutect-mutect2

Hope this helps.

[MurliNair]

Thanks Brad, I have set up the cancer/normal run again.

[MurliNair]

Brad, It ran to completion this time. I am studying the outputs. Thanks again for all your help. Hopefully I will be able to run my data using the pipeline. Cheers../Murli

The following are the last few lines of run. [2017-06-12T03:47Z] Storing in local filesystem: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/final/2014-08-13_dream-syn3/syn3-vardict-annotated.vcf.gz [2017-06-12T03:47Z] Storing in local filesystem: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/final/2014-08-13_dream-syn3/syn3-vardict-annotated.vcf.gz.tbi [2017-06-12T03:47Z] Storing in local filesystem: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/final/2014-08-13_dream-syn3/syn3-varscan-annotated.vcf.gz [2017-06-12T03:47Z] Storing in local filesystem: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/final/2014-08-13_dream-syn3/syn3-varscan-annotated.vcf.gz.tbi [2017-06-12T03:47Z] Storing in local filesystem: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/final/2014-08-13_dream-syn3/grading-summary-syn3.csv [2017-06-12T03:47Z] multiprocessing: upload_samples_project [2017-06-12T03:47Z] Storing in local filesystem: /mnt/workspaceXFS/mnair/research/CancerGenomics/genomeAnalysisTools/bcbioExample/cancer-dream-syn3/final/2014-08-13_dream-syn3/bcbio-nextgen.log [2017-06-12T03:47Z] Timing: finished

[MurliNair]

Hi Brad, I installed bcbio on the IU cluster locally in my account and tested it using run_tests.sh and it passed. I am now trying to set up a parallel run using the cancer/normal example. I studied the IPython Parallel section and wrote a script to submit the job using TORQUE. Could you check my script?

!/bin/tcsh

PBS -k o

PBS -l nodes=12:ppn=1:dc,vmem=2gb,walltime=24:00:00

PBS -M mnair@iusb.edu

PBS -m abe

PBS -j oe

module load java/1.7.0_51 bcbio_nextgen.py /N/u/mnair/Karst/workarea/bcbioRuns/cancer-dream-syn3/config/cancer-dream-syn3.yaml -t ipython -n 12 -s torque

[chapmanb]

Murli; Glad things are running smoothly locally and you're able to move to the cluster. For your submission script you only need to specify the resources used by the main bcbio driver script (a single node and core and a couple of gb of memory). bcbio itself does the work on allocating machine resources when running the actual work and will do so within the maximum number of cores you specify (12). So modifying your submission to:

 #PBS -l nodes=1:ppn=1:dc,vmem=4gb,walltime=24:00:00

would hopefully get it running for you. Hope this helps.

[MurliNair]

Thanks a lot Brad. Shall keep you posted.

[MurliNair]

Hi, I am running into problem when I am testing it on Mason which is IU main cluster for bioinformatics. I loaded the following modules before running run_test.sh module load java/1.7.0_51 module load python/2.7.9 I am attaching the output as a zipped file. nohup.zip

The last few lines are the following

CalledProcessError: Command '['bcbio_nextgen.py', '/gpfs/home/m/n/mnair/Mason/bcbio-nextgen/tests/test_automated_output/bcbio_system.yaml', '/gpfs/home/m/n/mnair/Mason/bcbio-nextgen/tests/data/automated/run_info-bam.yaml']' returned non-zero exit status 1

/N/u/mnair/Mason/local/share/bcbio/anaconda/lib/python2.7/subprocess.py:186: CalledProcessError ============================= 57 tests deselected ============================== ============= 2 failed, 9 passed, 57 deselected in 267.79 seconds ==============

As always appreciate your help.

[MurliNair]

For convenience I have posted the output here. http://bioinformatics.iusb.edu/errors/nohup.out

[chapmanb]

Thanks much for the detailed report. The error messages are coming from biobambam during sorting of BAM files after alignment and look like it's running out of memory. This is unexpected since these files are pretty small, but could be possible if you're running in a restricted environment with very little memory. Is it possible the machine you're running on has restricted policies on memory usage which cause this? Hope that helps explain the problem.

[MurliNair]

Thanks , I shall check the sys admin about it.

[MurliNair]

I tried to test it using TORQUE using the following

!/bin/tcsh

PBS -k o

PBS -l nodes=1:ppn=1,vmem=16gb,walltime=24:00:00

PBS -M mnair@iusb.edu

PBS -m abe

PBS -j oe

module load java/1.7.0_51 module load python/2.7.9 /N/u/mnair/Mason/bcbio-nextgen/tests/run_tests.sh

I have posted the errors here http://bioinformatics.iusb.edu/errors/qsubtest_sh_o490712.txt

I am not sure, if I needed anything else in the environment. Could you let me know if I missed anything?

[chapmanb]

Murli; Thanks for the additional details. This error happens when setting up the tests and looks like there is something problematic going on with your externally installed python. bcbio installs it's own python so I'd recommending avoiding the module load python/2.7.9 when running. Most module systems inject things into python paths which cause issues and I believe that is what is happening here. Hope that fixes it for you.

[MurliNair]

Brad, I ran it without loading the python module, I still get similar errors. http://bioinformatics.iusb.edu/errors/qsubtest_sh_o490938.txt I have sent the error to the sys admin of Mason and shared your comments as well. Thanks a ton for your time.

[MurliNair]

Hi Brad, I need your input once again. I am setting up bcbio on my new genomics server. The installation went fine as I followed all the guidelines you gave when I installed on my other server. I then ran the test script, which ran fine. The output is available here (http://bioinformatics.iusb.edu/errors/nohup_test1_3_18.txt) But when I ran the cancer normal I got an error complaining about the java version. The output is available here http://bioinformatics.iusb.edu/errors/nohup_cancerNormal_1_3_18.txt I would appreciate your input. Thanks ../Murli

[chapmanb]

Murli -- following up on this in your other duplicated issue: #2205