search

bcbio / bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis
https://bcbio-nextgen.readthedocs.io
MIT License
994 stars 354 forks source link

joblib.my_exceptions.JoblibAttributeError: JoblibAttributeError #2684

Closed amizeranschi closed 5 years ago

amizeranschi commented 5 years ago

Hello,

I'm getting several errors (below) when running joint VC with Strelka2 in bcbio-nextgen v.1.1.3 (stable). HaplotypeCaller works fine on my input data.

It also looks like Platypus isn't being found (if I try using it instead of HaplotypeCaller or Strelka2) and, indeed, it does look like it's missing from my Bcbio installation, although I remember that I've used it in the past -- has it been left out or is my bcbio installation incomplete?

Here is the YAML config file that I'm using:

details:
- algorithm:
    aligner: bwa
    effects: false
    jointcaller: strelka2-joint
    mark_duplicates: true
    ploidy: 2
    realign: false
    recalibrate: false
    tools_off:
    - gemini
    - contamination
    - peddy
    tools_on: gvcf
    variant_regions: ../config/variant_regions.bed
    variantcaller: strelka2
  analysis: variant2
  description: HG00096
  files:
  - /export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz
  - /export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz
  genome_build: hg38
  metadata:
    batch: testingVC
- algorithm:
    aligner: bwa
    effects: false
    jointcaller: strelka2-joint
    mark_duplicates: true
    ploidy: 2
    realign: false
    recalibrate: false
    tools_off:
    - gemini
    - contamination
    - peddy
    tools_on: gvcf
    variant_regions: ../config/variant_regions.bed
    variantcaller: strelka2
  analysis: variant2
  description: HG00097
  files:
  - /export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz
  - /export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz
  genome_build: hg38
  metadata:
    batch: testingVC
fc_name: testingVC-merged
upload:
  dir: ../final

And this is the outcome:

[2019-02-18T20:48Z] Resource requests: gatk, picard, strelka2; memory: 3.50, 3.00, 3.00; cores: 1, 16, 16
[2019-02-18T20:48Z] Configuring 2 jobs to run, using 1 cores each with 3.50g of memory reserved for each job
[2019-02-18T20:48Z] Timing: alignment post-processing
[2019-02-18T20:48Z] multiprocessing: piped_bamprep
[2019-02-18T20:48Z] Timing: variant calling
[2019-02-18T20:48Z] multiprocessing: variantcall_sample
Traceback (most recent call last):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin/bcbio_nextgen.py", line 238, in <module>
    main(**kwargs)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin/bcbio_nextgen.py", line 46, in main
    run_main(**kwargs)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 45, in run_main
    fc_dir, run_info_yaml)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 89, in _run_toplevel
    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 152, in variant2pipeline
    samples = genotype.parallel_variantcall_region(samples, run_parallel)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/variation/genotype.py", line 206, in parallel_variantcall_region
    "vrn_file", ["region", "sam_ref", "config"]))
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/split.py", line 35, in grouped_parallel_split_combine
    final_output = parallel_fn(parallel_name, split_args)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"], batch_size=1, backend="multiprocessing")(joblib.delayed(fn)(*x) for x in items):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 934, in __call__
    self.retrieve()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 862, in retrieve
    raise exception.unwrap(this_report)
joblib.my_exceptions.JoblibAttributeError: JoblibAttributeError
___________________________________________________________________________
...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin/bcbio_nextgen.py in <module>()
    233             if setup_info is None:  # no automated run after setup
    234                 sys.exit(0)
    235             workdir, new_kwargs = setup_info
    236             os.chdir(workdir)
    237             kwargs.update(new_kwargs)
--> 238         main(**kwargs)

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin/bcbio_nextgen.py in main(**kwargs={'config_file': 'bcbio_system.yaml', 'fc_dir': None, 'parallel': {'cores': 10, 'local_controller': False, 'module': 'bcbio.distributed', 'queue': None, 'resources': [], 'retries': 0, 'run_local': False, 'scheduler': None, 'tag': '', 'timeout': 15, ...}, 'run_info_yaml': '/export/automated-VC-test/testingVC-merged/config/testingVC-merged.yaml', 'workdir': '/export/automated-VC-test/testingVC-merged/work', 'workflow': None})
     41 from bcbio.graph import graph
     42 from bcbio.provenance import programs
     43 from bcbio.pipeline import version
     44 
     45 def main(**kwargs):
---> 46     run_main(**kwargs)
        kwargs = {'config_file': 'bcbio_system.yaml', 'fc_dir': None, 'parallel': {'cores': 10, 'local_controller': False, 'module': 'bcbio.distributed', 'queue': None, 'resources': [], 'retries': 0, 'run_local': False, 'scheduler': None, 'tag': '', 'timeout': 15, ...}, 'run_info_yaml': '/export/automated-VC-test/testingVC-merged/config/testingVC-merged.yaml', 'workdir': '/export/automated-VC-test/testingVC-merged/work', 'workflow': None}
     47 
     48 def parse_cl_args(in_args):
     49     """Parse input commandline arguments, handling multiple cases.
     50 

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py in run_main(workdir='/export/automated-VC-test/testingVC-merged/work', config_file='/export/bcbio_nextgen/galaxy/bcbio_system.yaml', fc_dir=None, run_info_yaml='/export/automated-VC-test/testingVC-merged/config/testingVC-merged.yaml', parallel={'cores': 10, 'local_controller': False, 'module': 'bcbio.distributed', 'queue': None, 'resources': [], 'retries': 0, 'run_local': False, 'scheduler': None, 'tag': '', 'timeout': 15, ...}, workflow=None)
     40     if config.get("log_dir", None) is None:
     41         config["log_dir"] = os.path.join(workdir, DEFAULT_LOG_DIR)
     42     if parallel["type"] in ["local", "clusterk"]:
     43         _setup_resources()
     44         _run_toplevel(config, config_file, workdir, parallel,
---> 45                       fc_dir, run_info_yaml)
        fc_dir = None
        run_info_yaml = '/export/automated-VC-test/testingVC-merged/config/testingVC-merged.yaml'
     46     elif parallel["type"] == "ipython":
     47         assert parallel["scheduler"] is not None, "IPython parallel requires a specified scheduler (-s)"
     48         if parallel["scheduler"] != "sge":
     49             assert parallel["queue"] is not None, "IPython parallel requires a specified queue (-q)"

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py in _run_toplevel(config={'algorithm': {}, 'bcbio_system': '/export/bcbio_nextgen/galaxy/bcbio_system.yaml', 'galaxy_config': 'universe_wsgi.ini', 'log_dir': '/export/automated-VC-test/testingVC-merged/work/log', 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, config_file='/export/bcbio_nextgen/galaxy/bcbio_system.yaml', work_dir='/export/automated-VC-test/testingVC-merged/work', parallel={'cores': 10, 'local_controller': False, 'module': 'bcbio.distributed', 'queue': None, 'resources': [], 'retries': 0, 'run_local': False, 'scheduler': None, 'tag': '', 'timeout': 15, ...}, fc_dir=None, run_info_yaml='/export/automated-VC-test/testingVC-merged/config/testingVC-merged.yaml')
     84     pipelines, config = _pair_samples_with_pipelines(run_info_yaml, config)
     85     system.write_info(dirs, parallel, config)
     86     with tx_tmpdir(config if parallel.get("type") == "local" else None) as tmpdir:
     87         tempfile.tempdir = tmpdir
     88         for pipeline, samples in pipelines.items():
---> 89             for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
        xs = undefined
        pipeline = <function variant2pipeline>
        config = {'algorithm': {}, 'bcbio_system': '/export/bcbio_nextgen/galaxy/bcbio_system.yaml', 'galaxy_config': 'universe_wsgi.ini', 'log_dir': '/export/automated-VC-test/testingVC-merged/work/log', 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}
        run_info_yaml = '/export/automated-VC-test/testingVC-merged/config/testingVC-merged.yaml'
        parallel = {'cores': 10, 'local_controller': False, 'module': 'bcbio.distributed', 'queue': None, 'resources': [], 'retries': 0, 'run_local': False, 'scheduler': None, 'tag': '', 'timeout': 15, ...}
        dirs = {'config': '/export/bcbio_nextgen/galaxy', 'fastq': None, 'flowcell': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}
        samples = [[{'algorithm': {'aligner': 'bwa', 'effects': False, 'jointcaller': 'strelka2-joint', 'mark_duplicates': True, 'ploidy': 2, 'realign': False, 'recalibrate': False, 'tools_off': ['gemini', 'contamination', 'peddy'], 'tools_on': 'gvcf', 'variant_regions': '../config/variant_regions.bed', ...}, 'analysis': 'variant2', 'description': 'HG00096', 'genome_build': 'hg38', 'metadata': {'batch': 'testingVC'}, 'resources': {}}], [{'algorithm': {'aligner': 'bwa', 'effects': False, 'jointcaller': 'strelka2-joint', 'mark_duplicates': True, 'ploidy': 2, 'realign': False, 'recalibrate': False, 'tools_off': ['gemini', 'contamination', 'peddy'], 'tools_on': 'gvcf', 'variant_regions': '../config/variant_regions.bed', ...}, 'analysis': 'variant2', 'description': 'HG00097', 'genome_build': 'hg38', 'metadata': {'batch': 'testingVC'}, 'resources': {}}]]
     90                 pass
     91 
     92 # ## Generic pipeline framework
     93 

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py in variant2pipeline(config={'algorithm': {}, 'bcbio_system': '/export/bcbio_nextgen/galaxy/bcbio_system.yaml', 'galaxy_config': 'universe_wsgi.ini', 'log_dir': '/export/automated-VC-test/testingVC-merged/work/log', 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, run_info_yaml='/export/automated-VC-test/testingVC-merged/config/testingVC-merged.yaml', parallel={'cores': 10, 'local_controller': False, 'module': 'bcbio.distributed', 'queue': None, 'resources': [], 'retries': 0, 'run_local': False, 'scheduler': None, 'tag': '', 'timeout': 15, ...}, dirs={'config': '/export/bcbio_nextgen/galaxy', 'fastq': None, 'flowcell': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, samples=[[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}]])
    147                     samples, config, dirs, "full",
    148                     multiplier=region.get_max_counts(samples), max_multicore=1) as run_parallel:
    149         with profile.report("alignment post-processing", dirs):
    150             samples = region.parallel_prep_region(samples, run_parallel)
    151         with profile.report("variant calling", dirs):
--> 152             samples = genotype.parallel_variantcall_region(samples, run_parallel)
        samples = [[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}]]
        run_parallel = <function run_parallel>
    153 
    154     ## Finalize variants, BAMs and population databases (per-sample multicore cluster)
    155     with prun.start(_wres(parallel, ["gatk", "gatk-vqsr", "snpeff", "bcbio_variation",
    156                                      "gemini", "samtools", "fastqc", "sambamba",

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/variation/genotype.py in parallel_variantcall_region(samples=[[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}]], run_parallel=<function run_parallel>)
    201     split_fn = _split_by_ready_regions(".vcf.gz", "work_bam", get_variantcaller)
    202     samples = _collapse_by_bam_variantcaller(
    203         grouped_parallel_split_combine(to_process, split_fn,
    204                                        multi.group_batches, run_parallel,
    205                                        "variantcall_sample", "concat_variant_files",
--> 206                                        "vrn_file", ["region", "sam_ref", "config"]))
    207     return extras + samples
    208 
    209 
    210 def vc_output_record(samples):

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/split.py in grouped_parallel_split_combine(args=[[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}]], split_fn=<function _do_work>, group_fn=<function group_batches>, parallel_fn=<function run_parallel>, parallel_name='variantcall_sample', combine_name='concat_variant_files', file_key='vrn_file', combine_arg_keys=['region', 'sam_ref', 'config'], split_outfile_i=-1)
     30       details.
     31     """
     32     grouped_args = group_fn(args)
     33     split_args, combine_map, finished_out, extras = _get_split_tasks(grouped_args, split_fn, file_key,
     34                                                                      split_outfile_i)
---> 35     final_output = parallel_fn(parallel_name, split_args)
        final_output = undefined
        parallel_fn = <function run_parallel>
        parallel_name = 'variantcall_sample'
        split_args = [[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00096-chr7_0_141973873.vcf.gz']]
     36     combine_args, final_args = _organize_output(final_output, combine_map,
     37                                                 file_key, combine_arg_keys)
     38     parallel_fn(combine_name, combine_args)
     39     return finished_out + final_args + extras

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py in run_parallel(fn_name='variantcall_sample', items=[[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00096-chr7_0_141973873.vcf.gz']])
     23         fn, fn_name = (fn_name, fn_name.__name__) if callable(fn_name) else (get_fn(fn_name, parallel), fn_name)
     24         logger.info("multiprocessing: %s" % fn_name)
     25         if "wrapper" in parallel:
     26             wrap_parallel = {k: v for k, v in parallel.items() if k in set(["fresources", "checkpointed"])}
     27             items = [[fn_name] + parallel.get("wrapper_args", []) + [wrap_parallel] + list(x) for x in items]
---> 28         return run_multicore(fn, items, config, parallel=parallel)
        fn = <function variantcall_sample>
        items = [[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00096-chr7_0_141973873.vcf.gz']]
     29     return run_parallel
     30 
     31 def get_fn(fn_name, parallel):
     32     taskmod = "multitasks"

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py in run_multicore(fn=<function variantcall_sample>, items=[[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00096-chr7_0_141973873.vcf.gz']], config={'algorithm': {}, 'bcbio_system': '/export/bcbio_nextgen/galaxy/bcbio_system.yaml', 'galaxy_config': 'universe_wsgi.ini', 'log_dir': '/export/automated-VC-test/testingVC-merged/work/log', 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, parallel={'cores': 10, 'cores_per_job': 1, 'local_controller': False, 'mem': '3.50', 'mem_pct': '1.00', 'module': 'bcbio.distributed', 'num_jobs': 2, 'progs': ['gatk', 'picard', 'variantcaller'], 'queue': None, 'resources': [], ...})
     81                                        max_multicore=int(parallel.get("max_multicore", sysinfo["cores"])))
     82     items = [config_utils.add_cores_to_config(x, parallel["cores_per_job"]) for x in items]
     83     if joblib is None:
     84         raise ImportError("Need joblib for multiprocessing parallelization")
     85     out = []
---> 86     for data in joblib.Parallel(parallel["num_jobs"], batch_size=1, backend="multiprocessing")(joblib.delayed(fn)(*x) for x in items):
        data = undefined
        parallel = {'cores': 10, 'cores_per_job': 1, 'local_controller': False, 'mem': '3.50', 'mem_pct': '1.00', 'module': 'bcbio.distributed', 'num_jobs': 2, 'progs': ['gatk', 'picard', 'variantcaller'], 'queue': None, 'resources': [], ...}
        fn = <function variantcall_sample>
        x = [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00096-chr7_0_141973873.vcf.gz']
        items = [[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'], [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': [...], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': [...]}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': [...]}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00096/samtools/HG00096.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00096/HG00096-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00096/HG00096-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00096-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00096/HG00096-variant_regions.regions.bed.gz'}}, 'description': 'HG00096', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00096_HG00096_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00096_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00096/HG00096-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00096-chr7_0_141973873.vcf.gz']]
     87         if data:
     88             out.extend(data)
     89     return out

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/joblib/parallel.py in __call__(self=Parallel(n_jobs=2), iterable=<generator object <genexpr>>)
    929                 # No need to wait for async callbacks to trigger to
    930                 # consumption.
    931                 self._iterating = False
    932 
    933             with self._backend.retrieval_context():
--> 934                 self.retrieve()
        self.retrieve = <bound method Parallel.retrieve of Parallel(n_jobs=2)>
    935             # Make sure that we get a last message telling us we are done
    936             elapsed_time = time.time() - self._start_time
    937             self._print('Done %3i out of %3i | elapsed: %s finished',
    938                         (len(self._output), len(self._output),

---------------------------------------------------------------------------
Joblib worker traceback:
---------------------------------------------------------------------------
AttributeError                                     Mon Feb 18 22:48:42 2019
PID: 41057Python 2.7.15: /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/bin/python
...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/joblib/parallel.py in __call__(self=<joblib.parallel.BatchedCalls object>)
    220     def __call__(self):
    221         # Set the default nested backend to self._backend but do not set the
    222         # change the default number of processes to -1
    223         with parallel_backend(self._backend, n_jobs=self._n_jobs):
    224             return [func(*args, **kwargs)
--> 225                     for func, args, kwargs in self.items]
        func = <function variantcall_sample>
        args = ({'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz')
        kwargs = {}
        self.items = [(<function variantcall_sample>, ({'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {...}, 'dexseq': {...}, 'express': {...}, 'gatk': {...}, 'macs2': {...}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {...}, 'seqcluster': {...}, 'snpeff': {...}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {...}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'), {})]
    226 
    227     def __len__(self):
    228         return self._size
    229 

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/utils.py in wrapper(*args=({'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'), **kwargs={})
     49 def map_wrap(f):
     50     """Wrap standard function to easily pass into 'map' processing.
     51     """
     52     @functools.wraps(f)
     53     def wrapper(*args, **kwargs):
---> 54         return f(*args, **kwargs)
        args = ({'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz')
        kwargs = {}
     55     return wrapper
     56 
     57 def transform_to(ext):
     58     """

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py in variantcall_sample(*args=({'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'))
    279 def get_parallel_regions(*args):
    280     return region.get_parallel_regions_block(*args)
    281 
    282 @utils.map_wrap
    283 def variantcall_sample(*args):
--> 284     return genotype.variantcall_sample(*args)
        args = ({'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, ('chr7', 0, 141973873), ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz')
    285 
    286 @utils.map_wrap
    287 def combine_variant_files(*args):
    288     return vcfutils.combine_variant_files(*args)

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/variation/genotype.py in variantcall_sample(data={'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}, region=('chr7', 0, 141973873), align_bams=['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], out_file='/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz')
    369             assert len(items) == len(align_bams)
    370         assoc_files = tz.get_in(("genome_resources", "variation"), data, {})
    371         if not assoc_files: assoc_files = {}
    372         for bam_file in align_bams:
    373             bam.index(bam_file, data["config"], check_timestamp=False)
--> 374         out_file = caller_fn(align_bams, items, ref_file, assoc_files, region, out_file)
        out_file = '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz'
        caller_fn = <function run>
        align_bams = ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam']
        items = [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}]
        ref_file = '/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/genomes/Hsapiens/hg38/seq/hg38.fa'
        assoc_files = {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}
        region = ('chr7', 0, 141973873)
    375     if region:
    376         data["region"] = region
    377     data["vrn_file"] = out_file
    378     return [data]

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/variation/strelka2.py in run(align_bams=['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], items=[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}], ref_file='/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/genomes/Hsapiens/hg38/seq/hg38.fa', assoc_files={'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, region=('chr7', 0, 141973873), out_file='/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873.vcf.gz')
     27     if paired:
     28         assert paired.normal_bam, "Strelka2 requires a normal sample"
     29         call_file = _run_somatic(paired, ref_file, assoc_files, region, call_file, strelka_work_dir)
     30     else:
     31         call_file = _run_germline(align_bams, items, ref_file,
---> 32                                   assoc_files, region, call_file, strelka_work_dir)
        assoc_files = {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}
        region = ('chr7', 0, 141973873)
        call_file = '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873-raw.vcf.gz'
        strelka_work_dir = '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873-work'
     33     return _af_annotate_and_filter(paired, items, call_file, out_file)
     34 
     35 def get_region_bed(region, items, out_file, want_gzip=True):
     36     """Retrieve BED file of regions to analyze, either single or multi-region.

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/variation/strelka2.py in _run_germline(align_bams=['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], items=[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}], ref_file='/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/genomes/Hsapiens/hg38/seq/hg38.fa', assoc_files={'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, region=('chr7', 0, 141973873), out_file='/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873-raw.vcf.gz', work_dir='/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873-work')
    136     return os.path.join(tx_work_dir, "runWorkflow.py")
    137 
    138 def _run_germline(align_bams, items, ref_file, assoc_files, region, out_file, work_dir):
    139     if not utils.file_exists(out_file):
    140         with file_transaction(items[0], work_dir) as tx_work_dir:
--> 141             workflow_file = _configure_germline(align_bams, items, ref_file, region, out_file, tx_work_dir)
        workflow_file = undefined
        align_bams = ['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam']
        items = [{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}]
        ref_file = '/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/genomes/Hsapiens/hg38/seq/hg38.fa'
        region = ('chr7', 0, 141973873)
        out_file = '/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873-raw.vcf.gz'
        tx_work_dir = '/export/automated-VC-test/testingVC-merged/work/bcbiotx/tmpk0gYHj/HG00097-chr7_0_141973873-work'
    142             _run_workflow(items[0], workflow_file, tx_work_dir)
    143         raw_file = os.path.join(work_dir, "results", "variants",
    144                                 "genome.vcf.gz" if joint.want_gvcf(items) else "variants.vcf.gz")
    145         utils.copy_plus(raw_file, out_file)

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/variation/strelka2.py in _configure_germline(align_bams=['/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam'], items=[{'align_bam': '/export/automated-VC-test/testingVC-merged/work/align/HG00097/HG00097-sort.bam', 'analysis': 'variant2', 'config': {'algorithm': {'adapters': [], 'align_split_size': None, 'aligner': 'bwa', 'archive': None, 'bam_clean': False, 'callable_count': 1, 'callable_regions': '/export/automated-VC-test/testingVC-merged/work/regions/testingVC-analysis_blocks.bed', 'coverage_interval': 'amplicon', 'effects': False, 'ensemble': None, ...}, 'resources': {'default': {'cores': 16, 'jvm_opts': ['-Xms750m', '-Xmx3500m'], 'memory': '3G'}, 'dexseq': {'memory': '10g'}, 'express': {'memory': '8g'}, 'gatk': {'jvm_opts': ['-Xms500m', '-Xmx3500m']}, 'macs2': {'memory': '8g'}, 'program_versions': '/export/automated-VC-test/testingVC-merged/work/provenance/programs.txt', 'qualimap': {'memory': '4g'}, 'seqcluster': {'memory': '8g'}, 'snpeff': {'jvm_opts': ['-Xms750m', '-Xmx3g']}}}, 'depth': {'bins': {'antitarget': None, 'seq2c': None, 'target': None}, 'samtools': {'idxstats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097-idxstats.txt', 'stats': '/export/automated-VC-test/testingVC-merged/work/qc/HG00097/samtools/HG00097.txt'}, 'sv_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/...00097/HG00097-sv_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/coverage/HG00097/HG00097-sv_regions.regions.bed.gz'}, 'variant_regions': {'dist': '/export/automated-VC-test/testingVC-merged/work/.../HG00097-variant_regions.mosdepth.region.dist.txt', 'regions': '/export/automated-VC-test/testingVC-merged/work/...ge/HG00097/HG00097-variant_regions.regions.bed.gz'}}, 'description': 'HG00097', 'dirs': {'fastq': None, 'galaxy': '/export/bcbio_nextgen/galaxy', 'work': '/export/automated-VC-test/testingVC-merged/work'}, 'files': ['/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R1.fastq.gz', '/export/automated-VC-test/testingVC-merged/work/align_prep/HG00097_HG00097_R2.fastq.gz'], 'files_orig': ['/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R1.fastq.gz', '/export/home/ncit/external/a.mizeranschi/automated-VC-test/testingVC-merged/HG00097_R2.fastq.gz'], 'genome_build': 'hg38', 'genome_resources': {'aliases': {'ensembl': 'homo_sapiens_merged_vep_95_GRCh38', 'human': True, 'snpeff': 'GRCh38.86'}, 'rnaseq': {'dexseq': '../rnaseq/ref-transcripts.dexseq.gff3', 'rRNA_fa': '../rnaseq/rRNA.fa', 'refflat': '../rnaseq/ref-transcripts.refFlat', 'transcriptome_index': {'tophat': '../rnaseq/tophat/hg38_transcriptome.ver'}, 'transcripts': '../rnaseq/ref-transcripts.gtf', 'transcripts_mask': '../rnaseq/ref-transcripts-mask.gtf'}, 'srnaseq': {'mint_lookup': '../srnaseq/LookupTable.tRFs.MINTmap_v1.txt', 'mint_other': '../srnaseq/OtherAnnotations.MINTmap_v1.txt', 'mint_space': '../srnaseq/tRNAspace.Spliced.Sequences.MINTmap_v1.fa', 'mirbase_hairpin': '../srnaseq/hairpin.fa', 'mirbase_mature': '../srnaseq/mature.fa', 'mirdeep2_fasta': '../srnaseq/Rfam_for_miRDeep.fa', 'srna_transcripts': '../srnaseq/srna-transcripts.gtf', 'trna_fasta': '../srnaseq/trna_mature_pre.fa'}, 'variation': {'1000g': None, 'clinvar': '/export/home/ncit/external/a.mizeranschi/bcbio_n...en/genomes/Hsapiens/hg38/variation/clinvar.vcf.gz', 'cosmic': '../variation/cosmic.vcf.gz', 'dbnsfp': '../variation/dbNSFP.txt.gz', 'dbscsnv': '../variation/dbscSNV.txt.gz', 'dbsnp': '/export/home/ncit/external/a.mizeranschi/bcbio_n.../genomes/Hsapiens/hg38/variation/dbsnp-151.vcf.gz', 'editing': '../editing/RADAR.bed.gz', 'encode_blacklist': None, 'esp': '../variation/esp.vcf.gz', 'exac': '../variation/exac.vcf.gz', ...}, 'version': 36}, ...}], ref_file='/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/genomes/Hsapiens/hg38/seq/hg38.fa', region=('chr7', 0, 141973873), out_file='/export/automated-VC-test/testingVC-merged/work/strelka2/chr7/HG00097-chr7_0_141973873-raw.vcf.gz', tx_work_dir='/export/automated-VC-test/testingVC-merged/work/bcbiotx/tmpk0gYHj/HG00097-chr7_0_141973873-work')
    121                                        "END=%s" % region[2], "CN"] + [str(x) for x in ploidies]) + "\n")
    122     return vcfutils.bgzip_and_index(out_file, items[0]["config"])
    123 
    124 def _configure_germline(align_bams, items, ref_file, region, out_file, tx_work_dir):
    125     utils.safe_makedir(tx_work_dir)
--> 126     cmd = [sys.executable, os.path.realpath(utils.which("configureStrelkaGermlineWorkflow.py"))]
        cmd = undefined
    127     cur_bed = get_region_bed(region, items, out_file)
    128     cmd += ["--referenceFasta=%s" % ref_file,
    129             "--callRegions=%s" % cur_bed,
    130             "--ploidy=%s" % _get_ploidy(shared.to_multiregion(region), items, out_file),

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/posixpath.py in realpath(filename=None)
    370 # filesystem).
    371 
    372 def realpath(filename):
    373     """Return the canonical path of the specified filename, eliminating any
    374 symbolic links encountered in the path."""
--> 375     path, ok = _joinrealpath('', filename, {})
        path = undefined
        ok = undefined
        filename = None
    376     return abspath(path)
    377 
    378 # Join two paths, normalizing and eliminating any symbolic links
    379 # encountered in the second path.

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/posixpath.py in _joinrealpath(path='', rest=None, seen={})
    376     return abspath(path)
    377 
    378 # Join two paths, normalizing and eliminating any symbolic links
    379 # encountered in the second path.
    380 def _joinrealpath(path, rest, seen):
--> 381     if isabs(rest):
        rest = None
    382         rest = rest[1:]
    383         path = sep
    384 
    385     while rest:

...........................................................................
/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/posixpath.py in isabs(s=None)
     49 # Return whether a path is absolute.
     50 # Trivial in Posix, harder on the Mac or MS-DOS.
     51 
     52 def isabs(s):
     53     """Test whether a path is absolute"""
---> 54     return s.startswith('/')
        s.startswith = undefined
     55 
     56 
     57 # Join pathnames.
     58 # Ignore the previous parts if a part is absolute.

AttributeError: 'NoneType' object has no attribute 'startswith'
chapmanb commented 5 years ago

Thanks for the report and sorry about the issue. Could you try updating to the latest bcbio development version:

bcbio_nextgen.py upgrade -u development

and re-running? We're working to move to Python 3 and the latest development version has some fixes which should help with finding these programs nested inside the python2 conda environment. Please let us know if there are still issues after updating, we're trying to get as much testing as possible with the changes before the next release.

amizeranschi commented 5 years ago

Looks like strelka2 was missing as well, never noticed it until you pointed out the migration to Python3. I ran the upgrade command, but couldn't find the missing tool binaries (strelka2, macs2, platypus etc.) in my $PATH after it finished. Then, I added the --tools option to the upgrade command. It mentioned that the tools were already installed:

# Installing into conda environment dv: deepvariant
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

# Installing into conda environment python2: cpat, dkfz-bias-filter, gemini, gvcf-regions, hap.py, lumpy-sv, macs2, manta, metasv, mirge, optitype, peddy, phylowgs, platypus-variant, sentieon, simple_sv_annotation, smcounter2, smoove, strelka, svtools, svtyper, theta2, tophat, vawk, vcf2db
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

# Installing into conda environment python3: atropos, crossmap
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

How are the different conda environments organized in the bcbio_nextgen directory structure and how will this affect the $PATH setup?

Indeed, I can see the respective package directories (e.g. platypus-variant-0.8.1.1-py27hdbffeaa_3 or macs2-2.1.2-py27r351_0) in bcbio_nextgen/anaconda/pkgs/. However, the corresponding executables don't appear in bcbio_nextgen/anaconda/bin/ or bcbio_nextgen/tools/bin/.

Actually, bcbio_nextgen/tools/bin/ only contains symlinks to the bcbio scripts: bcbio_fastq_umi_prep.py, bcbio_nextgen.py, bcbio_prepare_samples.py and bcbio_setup_genome.py.

Also, trying to manually run a binary directly from bcbio_nextgen/anaconda/pkgs/ results in an error. Is this to be expected?

/path_to/bcbio_nextgen/anaconda/pkgs/platypus-variant-0.8.1.1-py27hdbffeaa_3/bin/platypus 
/path_to/bcbio_nextgen/anaconda/pkgs/platypus-variant-0.8.1.1-py27hdbffeaa_3/bin/platypus: line 17: /opt/anaconda1anaconda2anaconda3/bin/python: No such file or directory
/path_to/bcbio_nextgen/anaconda/pkgs/macs2-2.1.2-py27r351_0/bin/macs2 
bash: /path_to/bcbio_nextgen/anaconda/pkgs/macs2-2.1.2-py27r351_0/bin/macs2: /opt/anaconda1anaconda2anaconda3/bin/python: bad interpreter: No such file or directory
chapmanb commented 5 years ago

Thanks for digging into this in depth and sorry for the continued problems. Since the packages you mention require python 2, they get installed into a separate conda environment: /path_to/bcbio_nextgen/anaconda/envs/python2/bin rather than the main anaconda/bin environment. However, they should get linked into the directory specified with --tooldir, which is the high level PATH you should include with bcbio. There are no symlinks to tools in that directory? I'm confused, as that should happen for everything in the main anaconda environment as well as the sub environments. It is possible the upgrade didn't complete successfully, or something else happened to prevent that? Could you provide the full command you're using to upgrade? Is this with the latest development version or 1.1.3? Thanks again for the help debugging.

amizeranschi commented 5 years ago

It's the latest development, I've run the upgrade a couple of times yesterday.

I found the tools in /path_to/bcbio_nextgen/anaconda/envs/python2/bin and running e.g. macs2 from there seems to work. However, the tools aren't getting linked inside /path_to/bcbio_nextgen/tools/bin/, which is what I had specified with --tooldir when I first installed bcbio. The only stuff in tools/bin are the four bcbio and bcbio_vm Python scripts.

I'm running upgrades like this: bcbio_nextgen.py upgrade -u development --data --tools --cwl. I've just run such an upgrade now and here is its full output:

bcbio_nextgen.py upgrade -u development --data --tools --cwl
Upgrading bcbio
Collecting package metadata: ...working... done
Solving environment: ...working... done

# All requested packages already installed.

WARNING: The --force flag will be removed in a future conda release.
         See 'conda install --help' for details about the --force-reinstall
         and --clobber flags.

Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda

  added / updated specs:
    - libgcc-ng

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

WARNING: The --force flag will be removed in a future conda release.
         See 'conda install --help' for details about the --force-reinstall
         and --clobber flags.

Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda

  added / updated specs:
    - libgcc-ng

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Upgrading bcbio-nextgen to latest development version
Collecting bcbio-nextgen from git+https://github.com/bcbio/bcbio-nextgen.git#egg=bcbio-nextgen
  Cloning https://github.com/bcbio/bcbio-nextgen.git to /tmp/pip-install-I0df9d/bcbio-nextgen
Building wheels for collected packages: bcbio-nextgen
  Running setup.py bdist_wheel for bcbio-nextgen ... done
  Stored in directory: /tmp/pip-ephem-wheel-cache-gAiVRO/wheels/c0/af/8f/75201c9fda71b27e67f797ef794dbade910690b2e3fa36c16c
Successfully built bcbio-nextgen
Installing collected packages: bcbio-nextgen
  Found existing installation: bcbio-nextgen 1.1.4a0
    Uninstalling bcbio-nextgen-1.1.4a0:
      Successfully uninstalled bcbio-nextgen-1.1.4a0
Successfully installed bcbio-nextgen-1.1.4a0
Upgrade of bcbio-nextgen development code complete.
## CWL support with bcbio-vm
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

Upgrading third party tools to latest versions
--2019-02-19 13:57:56--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com... 140.82.118.4, 140.82.118.3
Connecting to github.com|140.82.118.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following]
--2019-02-19 13:57:56--  https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master
Resolving codeload.github.com... 192.30.253.120, 192.30.253.121
Connecting to codeload.github.com|192.30.253.120|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘STDOUT’

     0K ........ ........ ........ ........ ........ ........ 3.24M
  3072K ........ ........ ........ .......                    13.7M=1.1s

2019-02-19 13:57:58 (4.62 MB/s) - written to stdout [5178650]

Setting up virtual machine
INFO: <cloudbio.flavor.Flavor instance at 0x7f3b12320b00>
INFO: This is a ngs_pipeline_minimal flavor
INFO: Reading default fabricrc.txt
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../config/fabricrc.txt
INFO: Distribution __auto__
INFO: Get local environment
INFO: CentOS setup
DBG [distribution.py]: NixPkgs: Ignored
[localhost] local: echo $HOME
[localhost] local: uname -m
INFO: Now, testing connection to host...
INFO: Connection to host appears to work!
DBG [utils.py]: Expand paths
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
INFO: Meta-package information from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
- Packages: minimal,libraries,perl,python,java,r,ruby,bio_nextgen
- Libraries: r-libs
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/custom.yaml
Reading packages from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/custom.yaml
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../config/chef_recipes.yaml
Reading packages from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../config/chef_recipes.yaml
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../config/puppet_classes.yaml
Reading packages from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../config/puppet_classes.yaml
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
INFO: Meta-package information from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
- Packages: minimal,libraries,perl,python,java,r,ruby,bio_nextgen
- Libraries: r-libs
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/packages-homebrew.yaml
Reading packages from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/packages-homebrew.yaml
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
INFO: Meta-package information from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/main.yaml
- Packages: minimal,libraries,perl,python,java,r,ruby,bio_nextgen
- Libraries: r-libs
DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/packages-conda.yaml
Reading packages from /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/packages-conda.yaml
Checking for problematic or migrated packages in default environment
# Installing into conda environment default: age-metasv, bamtools=2.4.0, bamutil, bbmap, bcbio-prioritize, bcbio-rnaseq, bcbio-variation, bcbio-variation-recall, bcftools, bedtools=2.27.1, bio-vcf, biobambam, bowtie, bowtie2, break-point-inspector, bwa, bwakit, cage, cancerit-allelecount, chipseq-greylist, cnvkit, coincbc, cramtools, cufflinks, cutadapt=1.16, cyvcf2, deeptools, delly, duphold>=0.0.9, ensembl-vep=95.*, express, extract-sv-reads, fastp, fastqc>=0.11.8=1, fgbio, freebayes=1.1.0.46, gatk, gatk4, geneimpacts, genesplicer, gffcompare, goleft, grabix, gridss, gsort, gvcfgenotyper, h5py, hisat2, hmftools-amber, hmftools-cobalt, hmftools-purple, hmmlearn, hts-nim-tools, htseq=0.9.1, htslib, impute2, kallisto>=0.43.1, kraken, lofreq, maxentscan, mbuffer, minimap2, mintmap, mirdeep2=2.0.0.7, mirtop, moreutils, mosdepth, multiqc, multiqc-bcbio, ngs-disambiguate, novoalign, octopus>=0.5.1b, oncofuse, parallel, pbgzip, perl-sanger-cgp-battenberg, picard, pindel, pizzly, preseq, pyloh, pysam>=0.14.0, pythonpy, qsignature, qualimap, rapmap, razers3=3.5.0, rtg-tools, sailfish, salmon, sambamba, samblaster, samtools, scalpel, seq2c, seqbuster, seqtk, sickle-trim, simple_sv_annotation, singlecell-barcodes, snap-aligner=1.0dev.97, snpeff=4.3.1t, solvebio, spades, star, stringtie, subread, survivor, tdrmapper, tophat-recondition, ucsc-bedgraphtobigwig, ucsc-bedtobigbed, ucsc-bigbedinfo, ucsc-bigbedsummary, ucsc-bigbedtobed, ucsc-bigwiginfo, ucsc-bigwigsummary, ucsc-bigwigtobedgraph, ucsc-bigwigtowig, ucsc-fatotwobit, ucsc-gtftogenepred, ucsc-liftover, ucsc-wigtobigwig, umis, vardict, vardict-java, variantbam, varscan, vcfanno, vcflib, verifybamid2, viennarna, vqsr_cnn, vt, wham, anaconda-client, awscli, bzip2, ncurses, nodejs, p7zip, readline, s3gof3r, xz, perl-app-cpanminus, perl-archive-extract, perl-archive-zip, perl-bio-db-sam, perl-cgi, perl-dbi, perl-encode-locale, perl-file-fetch, perl-file-sharedir, perl-file-sharedir-install, perl-ipc-system-simple, perl-lwp-protocol-https, perl-lwp-simple, perl-statistics-descriptive, perl-time-hires, perl-vcftools-vcf, bioconductor-annotate, bioconductor-biocgenerics, bioconductor-biocinstaller, bioconductor-biocstyle, bioconductor-biostrings, bioconductor-biovizbase, bioconductor-bsgenome.hsapiens.ucsc.hg19, bioconductor-bsgenome.hsapiens.ucsc.hg38, bioconductor-bubbletree, bioconductor-cn.mops, bioconductor-copynumber, bioconductor-degreport, bioconductor-deseq2, bioconductor-dexseq, bioconductor-dnacopy, bioconductor-genomeinfodbdata, bioconductor-genomicranges, bioconductor-iranges, bioconductor-limma, bioconductor-purecn, bioconductor-rtracklayer, bioconductor-titancna, r-base=3.4.1=h4fe35fd_8, r-basejump>=0.5.8, r-bcbiornaseq>=0.2.7, r-cghflasso, r-chbutils, r-devtools, r-dplyr, r-dt, r-ggdendro, r-ggplot2, r-ggrepel>=0.7, r-gplots, r-gsalib, r-knitr, r-pheatmap, r-plyr, r-pscbs, r-reshape, r-rmarkdown, r-rsqlite, r-sleuth, r-snow, r-stringi, r-viridis>=0.5, r-wasabi, r=3.4.1, xorg-libxt
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda

  added / updated specs:
    - age-metasv
    - anaconda-client
    - awscli
    - bamtools=2.4.0
    - bamutil
    - bbmap
    - bcbio-prioritize
    - bcbio-rnaseq
    - bcbio-variation
    - bcbio-variation-recall
    - bcftools
    - bedtools=2.27.1
    - bio-vcf
    - biobambam
    - bioconductor-annotate
    - bioconductor-biocgenerics
    - bioconductor-biocinstaller
    - bioconductor-biocstyle
    - bioconductor-biostrings
    - bioconductor-biovizbase
    - bioconductor-bsgenome.hsapiens.ucsc.hg19
    - bioconductor-bsgenome.hsapiens.ucsc.hg38
    - bioconductor-bubbletree
    - bioconductor-cn.mops
    - bioconductor-copynumber
    - bioconductor-degreport
    - bioconductor-deseq2
    - bioconductor-dexseq
    - bioconductor-dnacopy
    - bioconductor-genomeinfodbdata
    - bioconductor-genomicranges
    - bioconductor-iranges
    - bioconductor-limma
    - bioconductor-purecn
    - bioconductor-rtracklayer
    - bioconductor-titancna
    - bowtie
    - bowtie2
    - break-point-inspector
    - bwa
    - bwakit
    - bzip2
    - cage
    - cancerit-allelecount
    - chipseq-greylist
    - cnvkit
    - coincbc
    - cramtools
    - cufflinks
    - cutadapt=1.16
    - cyvcf2
    - deeptools
    - delly
    - duphold[version='>=0.0.9']
    - ensembl-vep=95
    - express
    - extract-sv-reads
    - fastp
    - fastqc[version='>=0.11.8',build=1]
    - fgbio
    - freebayes=1.1.0.46
    - gatk
    - gatk4
    - geneimpacts
    - genesplicer
    - gffcompare
    - goleft
    - grabix
    - gridss
    - gsort
    - gvcfgenotyper
    - h5py
    - hisat2
    - hmftools-amber
    - hmftools-cobalt
    - hmftools-purple
    - hmmlearn
    - hts-nim-tools
    - htseq=0.9.1
    - htslib
    - impute2
    - kallisto[version='>=0.43.1']
    - kraken
    - lofreq
    - maxentscan
    - mbuffer
    - minimap2
    - mintmap
    - mirdeep2=2.0.0.7
    - mirtop
    - moreutils
    - mosdepth
    - multiqc
    - multiqc-bcbio
    - ncurses
    - ngs-disambiguate
    - nodejs
    - novoalign
    - octopus[version='>=0.5.1b']
    - oncofuse
    - p7zip
    - parallel
    - pbgzip
    - perl-app-cpanminus
    - perl-archive-extract
    - perl-archive-zip
    - perl-bio-db-sam
    - perl-cgi
    - perl-dbi
    - perl-encode-locale
    - perl-file-fetch
    - perl-file-sharedir
    - perl-file-sharedir-install
    - perl-ipc-system-simple
    - perl-lwp-protocol-https
    - perl-lwp-simple
    - perl-sanger-cgp-battenberg
    - perl-statistics-descriptive
    - perl-time-hires
    - perl-vcftools-vcf
    - picard
    - pindel
    - pizzly
    - preseq
    - pyloh
    - pysam[version='>=0.14.0']
    - python=2
    - pythonpy
    - qsignature
    - qualimap
    - r-base==3.4.1=h4fe35fd_8
    - r-basejump[version='>=0.5.8']
    - r-bcbiornaseq[version='>=0.2.7']
    - r-cghflasso
    - r-chbutils
    - r-devtools
    - r-dplyr
    - r-dt
    - r-ggdendro
    - r-ggplot2
    - r-ggrepel[version='>=0.7']
    - r-gplots
    - r-gsalib
    - r-knitr
    - r-pheatmap
    - r-plyr
    - r-pscbs
    - r-reshape
    - r-rmarkdown
    - r-rsqlite
    - r-sleuth
    - r-snow
    - r-stringi
    - r-viridis[version='>=0.5']
    - r-wasabi
    - r=3.4.1
    - rapmap
    - razers3=3.5.0
    - readline
    - rtg-tools
    - s3gof3r
    - sailfish
    - salmon
    - sambamba
    - samblaster
    - samtools
    - scalpel
    - seq2c
    - seqbuster
    - seqtk
    - sickle-trim
    - simple_sv_annotation
    - singlecell-barcodes
    - snap-aligner=1.0dev.97
    - snpeff=4.3.1t
    - solvebio
    - spades
    - star
    - stringtie
    - subread
    - survivor
    - tdrmapper
    - tophat-recondition
    - ucsc-bedgraphtobigwig
    - ucsc-bedtobigbed
    - ucsc-bigbedinfo
    - ucsc-bigbedsummary
    - ucsc-bigbedtobed
    - ucsc-bigwiginfo
    - ucsc-bigwigsummary
    - ucsc-bigwigtobedgraph
    - ucsc-bigwigtowig
    - ucsc-fatotwobit
    - ucsc-gtftogenepred
    - ucsc-liftover
    - ucsc-wigtobigwig
    - umis
    - vardict
    - vardict-java
    - variantbam
    - varscan
    - vcfanno
    - vcflib
    - verifybamid2
    - viennarna
    - vqsr_cnn
    - vt
    - wham
    - xorg-libxt
    - xz

The following NEW packages will be INSTALLED:

  simple_sv_annotat~ bioconda/noarch::simple_sv_annotation-2019.02.18-py_0

The following packages will be UPDATED:

  blas                              pkgs/main::blas-1.0-mkl --> conda-forge::blas-1.1-openblas
  coincbc                                           2.9.8-1 --> 2.9.9-blas_openblash35bd731_1002
  gsl                       pkgs/main::gsl-2.2.1-h0c605f7_3 --> conda-forge::gsl-2.2.1-blas_openblash47a8a8e_5

The following packages will be SUPERSEDED by a higher-priority channel:

  numpy              pkgs/main::numpy-1.15.4-py27h7e9f1db_0 --> conda-forge::numpy-1.15.2-py27_blas_openblashd3ea46f_1
  scikit-learn       pkgs/main::scikit-learn-0.20.2-py27hd~ --> conda-forge::scikit-learn-0.20.0-py27_blas_openblasha84fab4_201
  scipy               pkgs/main::scipy-1.2.0-py27h7c811a0_0 --> conda-forge::scipy-1.1.0-py27_blas_openblash7943236_201

The following packages will be DOWNGRADED:

  numpy-base                          1.15.4-py27hde5b4d6_0 --> 1.15.4-py27h2f8d375_0

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
# Installing into conda environment dv: deepvariant
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

# Installing into conda environment python2: cpat, dkfz-bias-filter, gemini, gvcf-regions, hap.py, lumpy-sv, macs2, manta, metasv, mirge, optitype, peddy, phylowgs, platypus-variant, sentieon, smcounter2, smoove, strelka, svtools, svtyper, theta2, tophat, vawk, vcf2db
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

# Installing into conda environment python3: atropos, crossmap
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

# Installing into conda environment samtools0: ericscript
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

DBG [config.py]: Using config file /export/automated-VC-test/tmpbcbio-install/cloudbiolinux/cloudbio/../contrib/flavor/ngs_pipeline_minimal/r-libs.yaml
Creating manifest of installed packages in /export/bcbio_nextgen/manifest
Third party tools upgrade complete.
Upgrading bcbio-nextgen data files
--2019-02-19 14:07:54--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com... 140.82.118.4, 140.82.118.3
Connecting to github.com|140.82.118.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following]
--2019-02-19 14:07:55--  https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master
Resolving codeload.github.com... 192.30.253.120, 192.30.253.121
Connecting to codeload.github.com|192.30.253.120|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘STDOUT’

     0K ........ ........ ........ ........ ........ ........ 2.93M
  3072K ........ ........ ........ .......                    13.8M=1.2s

2019-02-19 14:07:56 (4.24 MB/s) - written to stdout [5178650]

Setting up virtual machine
INFO: <cloudbio.flavor.Flavor instance at 0x7f3b121d42d8>
INFO: <cloudbio.flavor.Flavor instance at 0x7f3b121d42d8>
INFO: This is a ngs_pipeline_minimal flavor
INFO: This is a ngs_pipeline_minimal flavor
INFO: Distribution __auto__
INFO: Distribution __auto__
INFO: Get local environment
INFO: Get local environment
INFO: CentOS setup
INFO: CentOS setup
WARN [distribution.py(206)]: NixPkgs are currently not supported for centos
WARN [distribution.py(206)]: NixPkgs are currently not supported for centos
DBG [distribution.py]: NixPkgs: Ignored
DBG [distribution.py]: NixPkgs: Ignored
[localhost] local: echo $HOME
[localhost] local: uname -m
INFO: Now, testing connection to host...
INFO: Now, testing connection to host...
INFO: Connection to host appears to work!
INFO: Connection to host appears to work!
DBG [utils.py]: Expand paths
DBG [utils.py]: Expand paths
List of genomes to get (from the config file at '{'install_liftover': False, 'genome_indexes': ['bwa', 'rtg'], 'genomes': [{'name': 'Human (hg38) full', 'validation': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'platinum-genome-NA12878', 'giab-NA12878-remap', 'giab-NA12878-crossmap', 'dream-syn4-crossmap', 'dream-syn3-crossmap', 'giab-NA12878-NA24385-somatic', 'giab-NA24143', 'giab-NA24149'], 'annotations': ['ccds', 'capture_regions', 'coverage', 'prioritize', 'dbsnp', 'hapmap_snps', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', '1000g_indels', 'clinvar', 'qsignature', 'genesplicer', 'effects_transcripts', 'varpon', 'vcfanno', 'viral'], 'dbkey': 'hg38', 'indexes': ['seq', 'twobit', 'bwa', 'hisat2']}], 'install_uniref': False}'): Human (hg38) full
bcbio-nextgen data upgrade complete.
Isolated tool installation not automatically added to environmental variables
 Add:
  /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin to PATH
Upgrade completed successfully.
amizeranschi commented 5 years ago

I've just thought of something that might be worth looking into.

I've made a parallel install of bcbio_nextgen on a cluster around the same time that I've created the problematic install that I mentioned here (which is on a cloud VM). I've just checked the cluster install now and the tools are all linked inside the bcbio_nextgen/tools/bin directory, including strelka2 and platypus et al.

When setting up the VM paths, I created a list of directories and a symlink in order to "mirror" the directory structure from the cluster, so that commands and scripts would work seamlessly on both.

My home directory is /export/home/ncit/external/a.mizeranschi. I had an "/export" mount point available on the VM and I had already copied a bunch of stuff there before getting access to the cluster, so I thought I'd create the /export/home/ncit/external/ directories and make a symlink there called a.mizeranschi, which points to the parent directory /export, such that the paths /export/<stuff> from the VM and and /export/home/ncit/external/a.mizeranschi/<stuff> from the cluster are analogous.

Could this setup with a smylink to a parent directory cause the problems I've been seeing? I haven't noticed any other strange things besides the issues with the tooldir in bcbio.

I've created a new bcbio install on the VM (where the symlink setup is) and noticed some errors during the install:

Installing bcbio-nextgen
Installing data and third party dependencies
Traceback (most recent call last):
  File "bcbio_nextgen_install.py", line 287, in <module>
    main(parser.parse_args(), sys.argv[1:])
  File "bcbio_nextgen_install.py", line 44, in main
    subprocess.check_call([bcbio, "upgrade"] + _clean_args(sys_argv, args))
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/bin/bcbio_nextgen.py', 'upgrade', '--tooldir=/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools', '--genomes', 'sacCer3', '--datatarget', 'variation', '--datatarget', 'rnaseq', '--datatarget', 'smallrna', '--aligners', 'bwa', '--aligners', 'bowtie2', '--aligners', 'hisat2', '--aligners', 'star', '--isolate', '--cores', '10', '--data']' returned non-zero exit status 1

 -- Upgrading with the latest development code and installing CWL support

Upgrading bcbio
Traceback (most recent call last):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin/bcbio_nextgen.py", line 221, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/install.py", line 56, in upgrade_bcbio
    args = add_install_defaults(args)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python2.7/site-packages/bcbio/install.py", line 706, in add_install_defaults
    raise ValueError("Default tool directory not yet saved in config defaults. "
ValueError: Default tool directory not yet saved in config defaults. Specify the '--tooldir=/path/to/tools' to upgrade tools. After a successful upgrade, the '--tools' parameter will work for future upgrades.

I ended up deleting the symlink, creating a directory instead and moving all the stuff there, as I probably should have done in the first place. After reinstalling bcbio_nextgen (with the same commands and paths), it finished alright and the tools are all linked inside bcbio_nextgen/tools/bin.

Sorry for wasting your time with this.

chapmanb commented 5 years ago

Thanks for following up on this. Apologies, some deeper symlinking approaches might break the logic, and it looks like we didn't have a tooldir in this specific case so it failed and never ended up fixing the symlinks.

I appreciate you working around the problem for your case and for the detailed description in case anyone else hits the same issue. Thanks again for all the work debugging this.