In crypto genome analysis pipeline, the first command "download" prepares the reference file, so need to be run firstly. After that, you can start the basis analsis (command 2-5) for each individual isolate genome. We recommend you always run command 2 "quality control" as the first step of the basic analysis. Command 3 "assembly" is a relatively independent step, while command 2,4,5 need to be run sequentially. After all the isolate genomes go through basis analysis individually, they will be ready for the independent advanced analysis (command 6-10) as a whole.
The scripts downloads the latest genome dna fasta and gff3 file and index fasta file:
download_and_index_genome_files.py -p $full_dir/properties.txt -g cryptosporidium_hominis
output files:
The script provides quality control on fastq files using trim_galore and remove duplicated reads using clumpify. The quality of the original and filtered reads are monitored by fastQC and visualized by multiQC.
quality_control.py -p $full_dir/properties.txt -fq1 $full_dir/ERR2889329_1.fastq -fq2 $full_dir/ERR2889329_2.fastq -pre test -de -m $full_dir/map_hominis_genotype_A10G2.txt
output files
single_end:
paired_end:
The script assembles short reads by using spades and provides statistics summary based on the result assemblies by using QUAST, and visualized by multiQC.
assembly.py -p $full_dir/properties.txt -g cryptosporidium_hominis -fq1 $full_dir/ERR2889329_1.fastq -fq2 $full_dir/ERR2889329_2.fastq -pre test -m $full_dir/map_hominis_genotype_A10G2.txt
output files:
The script mapping short reads to reference genomes using BWA or bowtie2, and creates statistics summary files using fastQC, qualiMap, and multiQC
reference mapping by using bwa:
reference-mapping_27Aug19.py -p $full_dir/properties.txt -t bwa -g cryptosporidium_hominis -fq1 $full_dir/ERR2889329_1_val_1.fq -fq2 $full_dir/ERR2889329_2_val_2.fq -pre xin_test_bwa_pair_ERR970586_trimmed -de -f illumina -l hominis
reference mapping by using bowtie2:
reference-mapping.py -p $full_dir/properties.txt -t bowtie2 -g cryptosporidium_hominis -fq1 $full_dir/ERR2889329_1_val_1.fq -fq2 $full_dir/ERR2889329_2_val_2.fq -pre test24Sep19 -f illumina -l hominis -m $full_dir/map_hominis_genotype_A10G2.txt
reference mapping by using bowtie2 for recombination
reference-mapping.py -p $full_dir/properties.txt -t bowtie2 -g cryptosporidium_hominis -fq1 $full_dir/ERR2889329_1_val_1.fq -fq2 $full_dir/ERR2889329_2_val_2.fq -pre test24Sep19 -f illumina -l hominis -m $full_dir/map_hominis_genotype_A10G2.txt -recom
output files:
The script creates filtered or unfiltered SNP and INDEL vcf and gvcf files from bam files using gatk, and then creates statistics summary by using bcf_tools and multiQC
variation_gatk.py -p $full_dir/properties.txt -g cryptosporidium_hominis -bam $full_dir/ERR2889329.dedup.bam -f -pre test -m $full_dir/map_hominis_genotype_A10G2.txt
output files
After all above commands are run sequencially for each isolate, the following advanced analysis can be run on all isolates independently.
The script invests genes under selection pressure within species through dNdS. It creates variation annotation file for each vcf file, and gene variation annotation summary file based on all vcf files by using snpEff.
dNdS_within_species.py -p /$full_dir/properties.txt -g 'cryptosporidium_hominis' -vp '$full_dir/*.snp.vcf' -go $full_dir/hominis_go_26Jul2019.csv -m $full_dir/map_hominis_genotype_A10G2.txt -pre 28Iso
output files:
The script creates the genome Short Tandem Repeat (STR) variation summary file based on all vcf files and creates multiple alignment files for all repeat regions.
get_repeat_variations.py -p $full_dir/properties.txt -g cryptosporidium_hominis -vp '$full_dir/*.vcf' -pre 28Iso -m $full_dir/map_hominis_genotype_A10G2.txt
output files
The script creates all individual chromosome multiple alignment for recombination.
build_chr_multiAlign_for_recombi.py -p $full_dir/properties.txt -g cryptosporidium_hominis -bp '$full_dir/*.bam' -pre forSimone -m $full_dir/map_hominis_genotype_A10G2.txt
output files:
a) prepare gvcf files for creating image:
The script merges gvcf files, seperating the variations into SNP and INDEL,and then do filtering if requested.
variation_gatk_gvcf.py -p $full_dir/properties.txt -g cryptosporidium_hominis -gv '$full_dir/*.snps.indels.g.vcf' -f -pre 28Iso
b) Create images
The script creates the following images: phylogeny tree, PCA, heatmap, upset, and SNP distribution on chromosome for vcf files
create_all_images.py -p /$full_dir/properties.txt -g cryptosporidium_hominis -v '$full_dir/*.snp.vcf' -gv '$full_dir/*.snp.g.vcf' -gt $full_dir/28Iso_map_for_visua.txt -pre 28Iso
output files:
The script creates multiQC html file based on fastqc, bcftools, snpEff, QUAST, and QualiMap output files. This script can only be run after the all isolate genomes go through commands 1-6.
multiQC.py -p $full_dir/properties.txt -pre test
output files: