Open ctb opened 1 month ago
TODO: make setup instructions / files available somewhere!
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --set channel_priority strict
conda env create -n smash-foo -f /opt/shared/sourmash-data/env-smash.yml
cat > env-smash.yml <<EOF
name: smash
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- sourmash>=4.8.10,<5
- seaborn
- scikit-learn
- seaborn
- sourmash_plugin_branchwater==0.9.5
- pip
- pip:
- sourmash_plugin_betterplot
- sourmash_plugin_abundhist
EOF
PatientHNC_06_illumina
curl -JLO ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR794/008/SRR7947178/SRR7947178_1.fastq.gz
curl -JLO ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR794/008/SRR7947178/SRR7947178_2.fastq.gz
PatientHNC_03_illumina
curl -JLO ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR794/009/SRR7947179/SRR7947179_1.fastq.gz
curl -JLO ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR794/009/SRR7947179/SRR7947179_2.fastq.gz
PatientHNC_10_illumina
curl -JLO ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR794/001/SRR7947181/SRR7947181_1.fastq.gz
curl -JLO ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR794/001/SRR7947181/SRR7947181_2.fastq.gz
sourmash sketch dna /opt/shared/sourmash-data/ihmp-shotgun-data/SRR7947178_?.fastq.gz -p k=31,abund -o SRR7947178.sig.zip --name SRR7947178
sourmash sketch dna /opt/shared/sourmash-data/ihmp-shotgun-data/SRR7947179_?.fastq.gz -p k=31,abund -o SRR7947179.sig.zip --name SRR7947179
sourmash sketch dna /opt/shared/sourmash-data/ihmp-shotgun-data/SRR7947181_?.fastq.gz -p k=31,abund -o SRR7947181.sig.zip --name SRR7947181
thoughts for next year -
https://hackmd.io/9ORFRJGaTOiOdEAY-Aih2A?view
open lab / Sat + Monday , Jul 20 & 22, 2024 / STAMPS 2024
Suggestion for open lab:
sourmash sketch dna
creates files containing DNA k-mers;Getting started
Start up RStudio on your instance, and click on Terminal.
Setup (do these once)
Create a working directory
Install sourmash and various plugins with conda (also see a conda tutorial; you'll need to install miniforge if you're not running conda/mamba on your Jetstream computer.)
Then run:
This installs:
Change directory and activate environment (each time you log in)
Change to sourmash working directory and activate sourmash software environment:
Might as well check for updates, sourmash is fast moving ;).
Convert some metagenomes into k-mer signatures
Use sourmash sketch to turn metagenomes into k-mers:
This will take about 5 minutes and create three
SRR*.sig.zip
files.(You can grab any metagenome you want from ENA, e.g. SRR7947181; use
curl -JLO <read URL
to download the FASTQ file(s) and sketch them as above!)Sketch some genomes into k-mer signatures
Use sourmash sketch to turn genomes into k-mers:
Run various analyses on the k-mer sketches!
Genome comparisons via
venn
andupset
Run:
to produce a file
genomes-venn.png
:Run:
to produce a file
genomes-upset.png
:Genome distance matrix, dendrograms, and ordination
First compare the genomes:
Now build a matrix+dendrogram view:
to produce
10sketches.mat.png
:You can produce a metric MDS plot too, colored by species:
to produce
10sketches.mds.png
:Flat (no abund) metagenome comparisons via
venn
andupset
Run a venn comparison of metagenomes =>
metag-venn.png
.Run an upset comparison of metagenomes =>
metag-upset.png
Genome presence/absence
Calculate which GTDB genomes are in a metagenome:
(will take ~3 minutes)
Make the detection/abundance plot:
=>
SRR7947178.detection.png
:=>
SRR7947178.ani.png
The three columns being plotted from
SRR7947178.x.gtdb-rs214.csv
are:f_match_orig
- the detection / containment of the match in the metagenome;match_containment_ani
- the containment-based ANI of the match;average_abund
- abundance of matching k-mers in the metagenome;Generating taxonomic classifications for metagenomes with sourmash
The basic workflow is to first run
sourmash gather
as above, and then runsourmash tax metagenome
.Run gather for each metagenome:
Then you can run taxonomic analyses like so:
See the sourmash tax documentations for various output options!
Getting % of metagenome covered from gather
If you've run a
fastgather
above, you will still need to runsourmash gather
to get nice human-readable output; you can do this quickly by using the fastgather output as a picklist to limit the results to the previous calculated output:and you should then see at the end:
Running abundhist to generate metagenome abundance histograms
After activating the
smash
conda environment, you can install the abundhist plugin like so:and run it on any metagenome or mixture that has been sketched with
-p abund
like so:Other topics we can discuss!
Appendix 1: UNIX Command Line!
Totally new to the command-line, or want to strengthen your foundation? Do this Unix Crash Course.
If you want some experience in other aspects, consider going through these:
Appendix 2: Conda tutorial
Please see my conda tutorial; you'll need to install miniforge if you're not running things on your Jetstream computer.
Appendix 3: examining assembly overlap with k-mers
See my in-class notes.
Contact info
Contact Titus at ctbrown@ucdavis.edu.