This PR closes #276 by introducing the TBProfiler_tNGS_PHB workflow, designed for Illumina PE tNGS data.
🗑️ This dev branch should NOT be deleted after merging to main.
:brain: Aim, Context and Functionality
tNGS is being used to analyze Mycobacterium tuberculosis data for clinical usage. Targeted sequence requires different analysis approaches to WGS, which means that TheiaProk workflows cannot be used as they are intended to create an assembled genome. Since this data is fragmented and amplicon-based, creating an assembly is a bad idea.
TBProfiler_tNGS_PHB is our solution: a workflow that performs minimal QC and runs TBProfiler and tbp-parser by default.
The minimal QC performed is as follows:
trimmomatic is run using a workflow parameter bases_to_crop (default=30) which cuts 30bp from the start and all bases that fall after a (average_read_length - 30bp) limit in the hope to remove primers and other sequencing artifacts.
we cannot trim primers because the method to generate this data uses proprietary primers
clockwork is currently not implemented due to difficult to resolve issues experienced during implementation of the tool.
:hammer_and_wrench: Impacted Workflows/Tasks & Changes Being Made
This will affect the behavior of the workflow(s) even if users don’t change any workflow inputs relative to the last version : No
Running this workflow on different occasions could result in different results, e.g. due to use of a live database, "latest" docker image, or stochastic data processing : No
:clipboard: Workflow/Task Step Changes
🔄 Data Processing
Docker/software or software versions changed:
tbp-parser has been updated to v1.3.0 which includes tNGS compatibility via the inclusion of the tNGS primer region bed file.
Databases or database versions changed:
No database changes.
Data processing/commands changed:
A new input parameter trimmomatic_base_crop is added to the trimmomatic_pe task. This Integer variable, if provided, will trigger calculation of average read length and creation of new parameters for the trimmomatic task, specifically: HEADCROP and CROP.
HEADCROP:<int> indicates the number of bases to remove from the START of the read
CROP:<int> indicates the FINAL LENGTH of the read that will be kept from the start of the read; any bases after this length will be removed.
Average read length is used to determine the CROP value dynamically; the trimmomatic_base_crop value will be removed from the average read length. HEADCROP is set to equal trimmomatic_base_crop.
No other analysis changes have been made to TBProfiler and tbp-parser (other than updated tbp-parser version, description available in tbp-parser repository).
File processing changed:
No file processing changes.
Compute resources changed:
No compute resources changes.
➡️ Inputs
All inputs are new because this is a new workflow.
New required inputs:
File read1
File read2
String samplename
New optional inputs for tbp_parser task:
Int coverage threshold
Int cpu
Int disk_size
String docker
Int memory
Int min_depth
String operator
String sequencing_method
Boolean tbp_parser_debug
New optional inputs for tbprofiler task:
Int cov_frac_threshold
Int cpu
Int disk_size
String mapper
Float min_af
Float min_af_pred
Int min_depth
Boolean ont_data
File tbprofiler_custom_db
String tbprofiler_docker_image
Boolean tbprofiler_run_custom_db
String variant_caller
String variant_calling_params
New optional inputs for tbprofiler_tngs workflow:
Int bases_to_crop
New optional inputs for trimmomatic_pe task:
Int disk_size
String docker
Int threads
String trimmomatic_args
Int trimmomatic_minlen
Int trimmomatic_quality_trim_score
Int trimmomatic_window_size
New optional inputs for version_capture task:
String docker
String timezone
⬅️ Outputs
All outputs are new because this is a new workflow.
New outputs (in alphabetical order):
tbp_parser_average_genome_depth
tbp_parser_coverage_report
tbp_parser_docker
tbp_parser_genome_percent_coverage
tbp_parser_laboratorian_report_csv
tbp_parser_lims_report_csv
tbp_parser_looker_report_csv
tbp_parser_version
tbprofiler_dr_type
tbprofiler_main_lineage
tbprofiler_median_coverage
tbprofiler_num_dr_variants
tbprofiler_num_other_variants
tbprofiler_output_alignment_bai
tbprofiler_output_alignment_bam
tbprofiler_pct_reads_mapped
tbprofiler_report_csv
tbprofiler_report_json
tbprofiler_report_tsv
tbprofiler_resistance_genes
tbprofiler_sub_lineage
tbprofiler_tngs_wf_analysis_date
tbprofiler_tngs_wf_version
tbprofiler_version
trimmomatic_docker
trimmomatic_read1_trimmed
trimmomatic_read2_trimmed
trimmomatic_stats
trimmomatic_version
:test_tube: Testing
Test Dataset
This workflow only works on Mycobacterium tuberculosis
Command-line Testing with MiniWDL or Cromwell (optional)
Terra Testing
Suggested Scenarios for Reviewer to Test
Theiagen Version Release Testing (optional)
:microscope: Final Developer Checklist
[x] The workflow/task has been tested locally and results, including file contents, are as anticipated: Yes/No
[x] The workflow/task has been tested on Terra and results, including file contents, are as anticipated: Yes/No
[x] The CI/CD has been adjusted and tests are passing: Yes/No
This PR closes #276 by introducing the TBProfiler_tNGS_PHB workflow, designed for Illumina PE tNGS data.
🗑️ This dev branch should NOT be deleted after merging to main.
:brain: Aim, Context and Functionality
tNGS is being used to analyze Mycobacterium tuberculosis data for clinical usage. Targeted sequence requires different analysis approaches to WGS, which means that TheiaProk workflows cannot be used as they are intended to create an assembled genome. Since this data is fragmented and amplicon-based, creating an assembly is a bad idea.
TBProfiler_tNGS_PHB is our solution: a workflow that performs minimal QC and runs TBProfiler and tbp-parser by default.
The minimal QC performed is as follows:
trimmomaticis run using a workflow parameterbases_to_crop(default=30) which cuts 30bp from the start and all bases that fall after a (average_read_length - 30bp) limit in the hope to remove primers and other sequencing artifacts.clockworkis currently not implemented due to difficult to resolve issues experienced during implementation of the tool.:hammer_and_wrench: Impacted Workflows/Tasks & Changes Being Made
This will affect the behavior of the workflow(s) even if users don’t change any workflow inputs relative to the last version : No
Running this workflow on different occasions could result in different results, e.g. due to use of a live database, "latest" docker image, or stochastic data processing : No
:clipboard: Workflow/Task Step Changes
🔄 Data Processing
Docker/software or software versions changed: tbp-parser has been updated to v1.3.0 which includes tNGS compatibility via the inclusion of the tNGS primer region bed file.
Databases or database versions changed: No database changes.
Data processing/commands changed: A new input parameter
trimmomatic_base_cropis added to the trimmomatic_pe task. This Integer variable, if provided, will trigger calculation of average read length and creation of new parameters for the trimmomatic task, specifically: HEADCROP and CROP.HEADCROP:<int>indicates the number of bases to remove from the START of the readCROP:<int>indicates the FINAL LENGTH of the read that will be kept from the start of the read; any bases after this length will be removed.Average read length is used to determine the CROP value dynamically; the
trimmomatic_base_cropvalue will be removed from the average read length. HEADCROP is set to equaltrimmomatic_base_crop.No other analysis changes have been made to TBProfiler and tbp-parser (other than updated tbp-parser version, description available in tbp-parser repository).
File processing changed: No file processing changes.
Compute resources changed: No compute resources changes.
➡️ Inputs
All inputs are new because this is a new workflow.
New required inputs:
File read1File read2String samplenameNew optional inputs for
tbp_parsertask:Int coverage thresholdInt cpuInt disk_sizeString dockerInt memoryInt min_depthString operatorString sequencing_methodBoolean tbp_parser_debugNew optional inputs for
tbprofilertask:Int cov_frac_thresholdInt cpuInt disk_sizeString mapperFloat min_afFloat min_af_predInt min_depthBoolean ont_dataFile tbprofiler_custom_dbString tbprofiler_docker_imageBoolean tbprofiler_run_custom_dbString variant_callerString variant_calling_paramsNew optional inputs for
tbprofiler_tngsworkflow:Int bases_to_cropNew optional inputs for
trimmomatic_petask:Int disk_sizeString dockerInt threadsString trimmomatic_argsInt trimmomatic_minlenInt trimmomatic_quality_trim_scoreInt trimmomatic_window_sizeNew optional inputs for
version_capturetask:String dockerString timezone⬅️ Outputs
All outputs are new because this is a new workflow.
New outputs (in alphabetical order):
tbp_parser_average_genome_depthtbp_parser_coverage_reporttbp_parser_dockertbp_parser_genome_percent_coveragetbp_parser_laboratorian_report_csvtbp_parser_lims_report_csvtbp_parser_looker_report_csvtbp_parser_versiontbprofiler_dr_typetbprofiler_main_lineagetbprofiler_median_coveragetbprofiler_num_dr_variantstbprofiler_num_other_variantstbprofiler_output_alignment_baitbprofiler_output_alignment_bamtbprofiler_pct_reads_mappedtbprofiler_report_csvtbprofiler_report_jsontbprofiler_report_tsvtbprofiler_resistance_genestbprofiler_sub_lineagetbprofiler_tngs_wf_analysis_datetbprofiler_tngs_wf_versiontbprofiler_versiontrimmomatic_dockertrimmomatic_read1_trimmedtrimmomatic_read2_trimmedtrimmomatic_statstrimmomatic_version:test_tube: Testing
Test Dataset
Command-line Testing with MiniWDL or Cromwell (optional)
Terra Testing
Suggested Scenarios for Reviewer to Test
Theiagen Version Release Testing (optional)
:microscope: Final Developer Checklist
🎯 Reviewer Checklist
🗂️ Associated Documentation (to be completed by Theiagen developer)