search

healthyPlant / PhytoPipe

10 stars 1 forks source link

readme

# PhytoPipe: A Phytosanitary Pipeline for plant pathogen detection and diagnosis using RNA-seq data

PhytoPipe is an open-source bioinformatics pipeline for plant pathogen detection using RNA-seq data. The pipeline is written in Snakemake (Köster and Rahmann 2018), a workflow management system for the development of data analysis workflows. PhytoPipe is divided into read quality control, read taxonomic classification, de novo assembly, contig blast annotation, virus species selection, viral reference mapping, and summary report. PhytoPipe can detect plant pathogens: bacteria (in addition to low expressed phytoplasma), fungi, viruses, and viroids. PhytoPipe has been tested on RNA-seq data (Illumina single-end and paired-end reads of different lengths) from different quarantined plant species such as apple, pear, peach, potato, sweet potato, cassava, rice, sugarcane, and bamboo. PhytoPipe is being used by the USDA Plant Germplasm Quarantine Program (PGQP) as the cornerstone of HTS-based diagnostics.

Workflow

scheme of workflow

Tools in pipeline

  1. Convert Blc to FastQ files (bcl2fastq, optional)
  2. Read QC (FastQC)
  3. Read preprocessing
    1. Host ribosomal RNA removal (bbduk.sh in BBTools)
    2. PCR duplicates removal (clumpify.sh in BBTools)
    3. Control/spike-in removal (bbsplit.sh in BBTools)
    4. Adaptor and read trimming (Trimmomatic)
  4. Read classification
    1. Nucleotide level (Kraken2)
    2. Protein level (Kaiju)
    3. View result with Krona pie chart (KronaTools)
  5. Assembly-based annotation
    1. Removal of host reads (modified KrakenTools (included) and seqtk)
    2. De novo assembly tools (Trinity || SPAdes)
    3. Assembly assessment report (QUAST)
    4. Blastn to viral reference genome or plant viral database or NCBI nt database (blastn)
    5. Diamond blastx to RVDB or NCBI nr database (Diamond)
    6. View conig classification with Krona pie chart (KronaTools)
  6. Reference-based mapping
    1. Read alignment (BWA || Bowtie2)
    2. Sort and index alignments (SAMtools)
    3. Consensus callers (BCFTools and BEDTools)
    4. Make pseudo contig (modified order_and_orient with MUMmer4 in viral-NGS (included))
  7. Present multisample QC and visualisation for raw read and assembly (MultiQC)

Quick start

Installation

For a Windows system, please use PhytoPipe docker image.

For a Linux or Mac system, you can download our code or clone the repository using Git:

cd /path/to/software
git clone https://github.com/healthyPlant/PhytoPipe.git

Then install dependencies. For an Ubuntu system, simply run

sudo bash /path/to/PhytoPipe/scripts/installTools.sh /path/to/software

For other Linux or Mac systems, please follow PhytoPipe wiki to install dependencies.

Since some tools conflict in a conda environment and are hard to update, we do not recommend installing all dependencies using conda. For a conda user, please check PhytoPipe wiki to see how to create a PhytoPipe conda environment.

Build databases

To install/update all databases, simply run

nohup bash /path/to/PhytoPipe/scripts/updateDatabase.sh /path/to/software/PhytoPipe /path/to/database v25.0 &

Please find the newest RVDB version from RVDB database, protein version and use the full path in the command. v25.0 in the above command is the RVDB version.

Set up configuration

Customize the workflow based on your needs in config.yaml, such as your sequence file extension and database paths. Please see the details in wiki.

Run PhytoPipe

Please check dependencies and databases requirements first using a dry-run (-n flag). If they are met, then you can run PhytoPipe

1. Run whole PhytoPipe

For fastq.gz reads input dry-run:

$ snakemake  --configfile /path/to/PhytoPipe/config.yaml -s /path/to/PhytoPipe/Snakefile --config workDir=/path/to/output/folder fastqDir=/path/to/input/fastq/folder --cores [number of cores ex. 16] -n

For raw Illumina flowcell's data input dry-run:

$ snakemake  --configfile /path/to/PhytoPipe/config.yaml -s /path/to/PhytoPipe/Snakefile --config workDir=/path/to/output/folder flowCellDir=/path/to/Illumina/run --cores [number of cores ex. 16] -n

If the dry-run succeeds, please remove '-n' parameter to run the pipeline. If you'd like to run it in the background, please use 'nohup'. For example:

$ nohup snakemake  --configfile /path/to/PhytoPipe/config.yaml -s /path/to/PhytoPipe/Snakefile --config workDir=/path/to/output/folder fastqDir=/path/to/input/fastq/folder --cores [number of cores ex. 16] &

Important: For workDir and fastqDir paths, full paths must be used.

You can view progress or errors in the file 'nohup.out' using the command

more nohup.out

2. Run partial PhytoPipe: NCBI nt and nr blast after the above viral pipeline

After finishing the virus detection using the virus database and RVDB, if you want to check NCBI nt and nr databases for other pathogens, you can try 

$ snakemake  --configfile your/config.yaml -s /path/to/PhytoPipe/Snakefile_runBlast --config workDir=/path/to/output/folder blastDbType=all --cores [number of cores ex. 16] -n

3. Run partial PhytoPipe: trim-assemble-annotation pipe

If your computer memory is limited (<256Gb), you can skip the classification steps (Kraken2 and Kaiju) to directly run trim-assemble-annotation, then check your results in the annotation folder (sample.blast.nx.txt, sample.blastnt0.txt, sample.blastnr0.txt). Here is a dry-run command example for running Trimmomatic trim, Spades assemble, and Blast against viral databases and NCBI nt/nr database annotation:

$ snakemake  --configfile /path/to/PhytoPipe/config.yaml -s /path/to/PhytoPipe/Snakefile_runAnnotation --config workDir=/path/to/output/folder fastqDir=/path/to/input/fastq/folder mapReadType=trimmed assembler=Spades blastDbType=all --cores [number of cores ex. 16] -n

Note: seq_type (se/pe), mapReadType, assembler, blastDbType ban be changed in config.yaml

4. Run partial PhytoPipe: trim-assemble-annotation pipe with only NCBI nt and nr databases

If you just want annotate the contigs using NCBI nt (directly download) and nr databases (Diamond built), you can run another trim-assemble-annotation pipeline, then check your results in the annotation folder (sample.blastnt0.txt, sample.blastnr0.txt). Here is a dry-run command example for running Trimmomatic trim, Spades assemble, and Blast against NCBI nt/nr database annotation:

$ snakemake  --configfile /path/to/PhytoPipe/config.yaml -s /path/to/PhytoPipe/Snakefile_runAnnotationNt --config workDir=/path/to/output/folder fastqDir=/path/to/input/fastq/folder mapReadType=trimmed assembler=Spades --cores [number of cores ex. 16] -n

PhytoPipe quick test

After the software and databases are ready, you can run a quick test using the VIROMOCKchallenge Dataset8, which is in the test/data folder. The steps can be found in test notebook.

PhytoPipe docker image

The PhytoPipe docker image can be pulled 

docker pull healthyplant/phytopipe

Docker can help you avoid manually installing the software. You can use the docker image on many systems (Linux, Mac, Windows). PhytoPipe docker image usage is in its docker README. PhytoPipe docker tutorial can be found in test docker.

Documentation

More information on input/output, dependencies, and databases are found in the wiki

Citation

Hu, X., Hurtado-Gonzales, O.P., Adhikari, B.N. et al. PhytoPipe: a phytosanitary pipeline for plant pathogen detection and diagnosis using RNA-seq data. BMC Bioinformatics 24, 470 (2023). https://doi.org/10.1186/s12859-023-05589-2

Contact

Alex Hu (xiaojun.hu (at) usda.gov)