Percent Gene Coverage Task Modularization

sage-wright commented 4 months ago

This PR closes #325 and closes #312

🗑️ This dev branch should be deleted after merging to main.

~Currently waiting on the organism parameter PR merging before adding this since it will use that workflow.~

:brain: Aim, Context and Functionality

As TheiaCoV expands to include more organisms, having a WDL task that is hard-coded for a single organism is inefficient if we want to mimic the behavior for other organisms. This PR changes the calculation of breadth of coverage to no longer be hard-coded and is now organism-agnostic. This requires the usage of the organism_parameter logic subworklow and also enables the user to specify the particular regions they want to be listed in the output file by overwriting the default.

Default bed files are currently only provided for mpox and SC2.

:hammer_and_wrench: Impacted Workflows/Tasks & Changes Being Made

This will affect the behavior of the workflow(s) even if users don’t change any workflow inputs relative to the last version : Yes

Output file now has a different disclaimer that is now applicable to all organisms, not just SC2

Running this workflow on different occasions could result in different results, e.g. due to use of a live database, "latest" docker image, or stochastic data processing : No

:clipboard: Workflow/Task Step Changes

🔄 Data Processing

The same calculations are used but now require a bed file as input to determine the regions of interest. This input bed file is looped through and samtools depth is used to determine the percentage of the sites that are above the specified minimum depth.

Docker/software or software versions changed:

Databases or database versions changed:

Data processing/commands changed:

File processing changed:

Compute resources changed:

➡️ Inputs

New input: reference_gene_locations_bed which indicates that the gene locations should correspond to the same reference file that was used for alignment. By default, this file is provided for SC2 and mpox. The user can use this input file to overwrite the defaults.

⬅️ Outputs

The sc2_all_genes_percent_coverage file is now est_percent_gene_coverage_tsv as it is no longer SC2 specific.

:test_tube: Testing

Test Dataset

Commandline Testing with MiniWDL or Cromwell (optional)

Terra Testing

[x] Illumina PE with mpox & sc2 here
[x] Illumina SE with all organisms here
[x] ONT with all organisms except HIV here
[x] ClearLabs here

Suggested Scenarios for Reviewer to Test

None of the workflows other than the ones mentioned above are actually changed. I just fixed some formatting with brackets and spacing because I saw them and needed to fix them immediately. They don't change any functionality at all.

Theiagen Version Release Testing (optional)

:microscope: Final Developer Checklist

[x] The workflow/task has been tested locally and results, including file contents, are as anticipated
[x] The workflow/task has been tested on Terra and results, including file contents, are as anticipated
[x] The CI/CD has been adjusted and tests are passing (to be completed by Theiagen developer)
[x] Code changes follow the style guide

🎯 Reviewer Checklist

[x] All impacted workflows/tasks have been tested on Terra with a different dataset than used for development
[x] All reviewer-suggested scenarios have been tested and any additional
[x] All changed results have been confirmed to be accurate
[x] All workflows/tasks impacted by change/s have been tested using a standard validation dataset to ensure no unintended change of functionality
[x] All code adheres to the style guide
[x] MD5 sums have been updated
[x] The PR author has addressed all comments