workshop 9: Automating your analyses with the snakemake workflow system

[ctb]

Wednesday August 25 from 9 am - 11:30 PDT

Instructors: Abhijna Parigi and Titus Brown Moderator: Marisa Helpers:

Zoom link:

Description:

This two hour workshop will introduce attendees to the snakemake workflow system, for executing large-scale automated analyses.

draft lesson: snakemake for workflows. https://github.com/ngs-docs/2021-GGG298/tree/latest/Week4-snakemake-for-workflows

owner: ???

[abhijna]

I can't assign myself yet, but happy to co-instruct this workshop :)

[ctb]

I cannot lead (may be a bit late to start). Pamela cannot make it.

[abhijna]

I will do the first half and Titus will do the second. I'm thinking I can end before the "Running Salmon quant" section. That way Titus can teach how to add one last rule and then talk about tips, best practices, etc.

Draft lesson is up -- feel free to leave comments here or on the PR!

[marisalim]

took a look at the notes so far, a few ideas :)

• section 9.6.1 says to use Rstudio editor - change to nano?
• maybe have them run the snakemake command specifying the all rule the first time. or say that by default when you run snakemake without specifying a rule, it chooses the all rule.
• it might help to show the process of running the commands for 1 sample first, and then show how it's translated into a snakemake workflow (run multiple steps with 1 command! run multiple samples with multiple steps with 1 command!)
• for 9.11, maybe introduce --dry-run and then say the shorthand is -n
• 9.12.2 - i think there's a missing new line here? how about putting the snakemake commands in backticks. there are a couple of places where I think you need to add a blank line for the markdown render
• would be great to see the full Snakefile code at the end of the lesson

[abhijna]

Suggested survey questions:

• I understand what a computational workflow is
• I can decorate a snakefile with inputs and outputs
• I can add new rules to a snakefile
• I can execute a snakemake workflow

[jeremywalter]

Pre-Survey: https://forms.gle/1wUXycHSPA4Gajt27 Post-Survey: https://forms.gle/jiSvrdHYUFBQdoBA9

[abhijna]

2021-remote-computing-9pptx.pptx

[nick-ulle]

Drake is called Targets now.

[marisalim]

suggestions for the future :)

• common syntax errors from having tab/spacing issues in the Snakefile rules. maybe introduce nano -ET4 Snakefile earlier.

[abhijna]

add multiple inputs to lesson

[marisalim]

this is the final snakefile with multiple inputs & expand()

SAMPLES=["ERR458493", "ERR458501", "ERR458494", "ERR458500"]
print('samples are:', SAMPLES)
rule all:
    input:
        expand("{sample}_fastqc.html", sample=SAMPLES),
        "orf_coding.fasta.gz",
        "yeast_orfs",
        expand("{sample}.quant", sample=SAMPLES),

rule make_fastqc:
    input:
        "{sample}.fastq.gz",
    output:
        "{sample}_fastqc.html",
        "{sample}_fastqc.zip"  
    shell:
        "fastqc {input}"

rule download_reference:
    output:
        "orf_coding.fasta.gz"
    shell:
        "curl -L -O https://downloads.yeastgenome.org/sequence/S288C_reference/orf_dna/orf_coding.fasta.gz"

rule index_reference:
    input:
        "orf_coding.fasta.gz"
    output:
        directory("yeast_orfs")
    shell:
        "salmon index --index yeast_orfs --transcripts {input}"

rule salmon_quant:
    input: 
        fastq = "{sample}.fastq.gz",
        index = "yeast_orfs"
    output: 
        directory("{sample}.quant")
    shell:
        "salmon quant -i {input.index} --libType U -r {input.fastq} -o {output} --seqBias --gcBias"