GATK Cohort Calling Stash Not Working

[imtiyazhariyani]

Tried running the GATK Cohort workflow from BioSAILS without running it per chromosome. The workflow executes and produces an s batch script but with no rules. Samples are found. However, the following line is displayed when I run the biox command:

"Path::Tiny paths require defined, positive-length parts at /scratch/gencore/.local/easybuild/software/gencore_dev/1.0/lib/perl5/site_perl/5.22.0/BioX/Workflow/Command/run/Rules/Directives/Types/Path.pm line 183."

Below is the yml script:

`--- global:

Initial Directory Setup

- indir:      "data/analysis_imtiyaz"
- outdir:     "data/analysis_imtiyaz/cohort"
# indir/outdir is a chained variable

it gets changed within a rule

- root_in_dir: "data/analysis"
- root_out_dir: "data/analysis/cohort"
# Find Samples
- sample_glob: "data/analysis_imtiyaz/Sample*/gatk/*_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
- by_sample_outdir: '1'
# Analysis Dirs
- combine_dir: "data/analysis_imtiyaz"
# Reference Data
- bwa_mem_reference: "/scratch/gencore/160713_SN7001341_0131_AC8YL9ACXX/Unaligned/Project_Boissinot_lab/data/analysis/reference/leptopelis_transcriptrinity"
- reference: "{$self->bwa_mem_reference}.fa"
# HPC Directives
- HPC:
   - account: 'ieh211'
   - partition: 'serial'
   - module:  'gencore gencore_dev gencore_variant_detection/1.0'
   - cpus_per_task: 1
   - commands_per_node: 1

rules:

• stash_samples: local:

  • override_process: 1
  • create_outdir: 0 process: |- { use File::Glob; use Cwd; my @glob = glob(cwd().'/'. $self->sample_glob); $self->stash->{sample_files} = \@glob; ($SILENTLY); }

• combine_gvcf: local:

  • override_process: 1
  • indir: "{$self->combine_dir}"
  • outdir: "{$self->combine_dir}"
  • OUTPUT: "{$self->outdir}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
  • HPC:
    • deps: 'stash_samples'
    • walltime: '48:00:00'
    • mem: '40GB'
  • process_mustache: | gatk -Xmx80G -T CombineGVCFs \ -R {{{reference}}} \ {{#stash.sample_files}} --variant {{{.}}} \ {{/stash.sample_files}} -o {{{outdir}}}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf process: |- { $OUT .= $self->render_mustache($self->process_mustache); }

• cohort_calling: local:

  • indir: "{$self->{combine_dir}"
  • outdir: "{$self->{combine_dir}"
  • INPUT: "{self->combine_dir}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
  • OUTPUT: "{$self->combine_dir}/ALL_SAMPLES_cohort.haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
  • process_mustache: | gatk -T GenotypeGVCFs \ -R {{{reference}}} \ -stand_call_conf '30' \ -o {{{outdir}}}/ALL_SAMPLES_cohort.haplotype.realigned.withrg.csorted.cleaned.aligned.vcf
  • HPC:
    • deps: 'combine_gvcf'
    • walltime: '48:00:00'
    • mem: '40GB' process: |- { $OUT .= $self->render_mustache($self->process_mustache); } `

[jerowe]

@imtiyazhariyani , let me check this out. Its not immediately apparent whats going on.

[jerowe]

Fixed - there were a few issues with the cohort calling rule.

- cohort_calling:
    local:
             # Should be {$self->combine_dir}
            - indir: "{$self->{combine_dir}"
            - outdir: "{$self->{combine_dir}"
            # Should be {$self-> ... }
            - INPUT: "{self->combine_dir}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"

Should be

      - cohort_calling:
          local:
              - indir: "{$self->combine_dir}"
              - outdir: "{$self->combine_dir}"
              - INPUT: "{$self->indir}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"

[jerowe]

Here's the whole thing.

---
global:
    # Initial Directory Setup
    - indir: "data/analysis_imtiyaz"
    - outdir: "data/analysis_imtiyaz/cohort"
    # indir/outdir is a chained variable
    - root_in_dir: "data/analysis"
    - root_out_dir: "data/analysis/cohort"
    # Find Samples
    - sample_glob: "data/analysis_imtiyaz/Sample*/gatk/*_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
    - by_sample_outdir: '1'
    # Analysis Dirs
    - combine_dir: "data/analysis_imtiyaz"
    # Reference Data
    - bwa_mem_reference: "/scratch/gencore/160713_SN7001341_0131_AC8YL9ACXX/Unaligned/Project_Boissinot_lab/data/analysis/reference/leptopelis_transcriptrinity"
    - reference: "{$self->bwa_mem_reference}.fa"
    # HPC Directives
    - HPC:
      - account: 'ieh211'
      - partition: 'serial'
      - module:  'gencore gencore_dev gencore_variant_detection/1.0'
      - cpus_per_task: 1
      - commands_per_node: 1
rules:
      - stash_samples:
          local:
              - override_process: 1
              - create_outdir: 0
          process: |-
              {
                use File::Glob;
                use Cwd;
                my @glob = glob(cwd().'/'. $self->sample_glob);
                $self->stash->{sample_files} = \@glob;
                ($SILENTLY);
              }
      - combine_gvcf:
          local:
              - override_process: 1
              - indir: "{$self->combine_dir}"
              - outdir: "{$self->combine_dir}"
              - OUTPUT: "{$self->outdir}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
              - HPC:
                   - deps: 'stash_samples'
                   - walltime: '48:00:00'
                   - mem: '40GB'
              - process_mustache: |
                 gatk -Xmx80G -T CombineGVCFs \
                 -R {{{reference}}} \
                 {{#stash.sample_files}}
                   --variant {{{.}}} \
                 {{/stash.sample_files}}
                 -o {{{outdir}}}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf
          process: |-
              {
                $OUT .= $self->render_mustache($self->process_mustache);
              }
      - cohort_calling:
          local:
              - indir: "{$self->combine_dir}"
              - outdir: "{$self->combine_dir}"
              - INPUT: "{$self->indir}/ALL_SAMPLES_haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
              - OUTPUT: "{$self->outdir}/ALL_SAMPLES_cohort.haplotype.realigned.withrg.csorted.cleaned.aligned.vcf"
              - process_mustache: |
                  gatk -T GenotypeGVCFs \
                    -R {{{reference}}} \
                    -stand_call_conf '30' \
                    -o {{{outdir}}}/ALL_SAMPLES _cohort.haplotype.realigned.withrg.csorted.cleaned.aligned.vcf
              - HPC:
                - deps: 'combine_gvcf'
                - walltime: '48:00:00'
                - mem: '40GB'
          process: |-
            {
              $OUT .= $self->render_mustache($self->process_mustache);
            }

[imtiyazhariyani]

Oops that was fairly straightforward! It works now, thank you @jerowe