update default pangolin docker (pdata 1.27) & nextclade dataset tags for SC2, Flu, Mpox

[kapsakcj]

This PR closes #454

🗑️ This dev branch should be deleted after merging to main.

:brain: Aim, Context and Functionality

To update the default docker image for pangolin

& to update the default nextclade dataset tags to the latest available at the current time

:hammer_and_wrench: Impacted Workflows/Tasks & Changes Being Made

This will affect the behavior of the workflow(s) even if users don’t change any workflow inputs relative to the last version : Yes

Running this workflow on different occasions could result in different results, e.g. due to use of a live database, "latest" docker image, or stochastic data processing : No

Changes were made to the organism_parameters.wdl subworkflow

• Impacts all TheiaCoV workflows that call Pangolin or Nextclade on SC2, Mpox, and Influenza
• Theiacov_fasta_batch
• Pangolin_Update
• (indirectly) Augur since it imports the organism_parameters subwf

:clipboard: Workflow/Task Step Changes

🔄 Data Processing

Docker/software or software versions changed: same pangolin version, pangolin-data updated to v1.27

Databases or database versions changed: updated dataset tags for SC2, Mpox, various Flu types & subtypes

Data processing/commands changed: N/A

File processing changed: N/A

Compute resources changed: N/A

➡️ Inputs

N/A

⬅️ Outputs

:test_tube: Testing

Test Dataset

sars-cov-2

Flu A and B samples

Mpox

Commandline Testing with MiniWDL or Cromwell (optional)

Not tested locally

Terra Testing

• SC2 samples via theiacov_fasta_batch: https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/5de63475-830b-437c-ad53-8963a6009eab
  • ✅ wf used the updated 2024-04-19 dataset tag as expected
• Flu A and B samples via theiacov_illumina_PE: https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/1ee38cb5-08cf-437e-959c-58a64c896f00
  • ✅ wf used the updated 2024-04-19 dataset tag as expected
• Mpox samples via theiacov_illumina_PE: https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/69c27521-3ce5-4056-a019-04c3afe8a555
  • ✅ wf used the updated 2024-04-19 dataset tag as expected

⚠️ Need to review outputs for all of these ^

Suggested Scenarios for Reviewer to Test

Test the various organisms through various TheiaCoV workflows (or pangolin_update)

Theiagen Version Release Testing (optional)

• Will changes require functional or validation testing (checking outputs etc) during the release? functional
• Do new samples need to be added to validation datasets? If so, upload these to the appropriate validation workspace Google bucket (). Please describe the new samples here and why these have been chosen. No
• Are there any output files that should be checked after running the version release testing? No

:microscope: Final Developer Checklist

• [x] The workflow/task has been tested locally and results, including file contents, are as anticipated
• [x] The workflow/task has been tested on Terra and results, including file contents, are as anticipated
• [x] The CI/CD has been adjusted and tests are passing (to be completed by Theiagen developer)
• [x] Code changes follow the style guide

🎯 Reviewer Checklist

• [ ] All impacted workflows/tasks have been tested on Terra with a different dataset than used for development
• [ ] All reviewer-suggested scenarios have been tested and any additional
• [ ] All changed results have been confirmed to be accurate
• [ ] All workflows/tasks impacted by change/s have been tested using a standard validation dataset to ensure no unintended change of functionality
• [ ] All code adheres to the style guide
• [ ] MD5 sums have been updated
• [ ] The PR author has addressed all comments

🗂️ Associated Documentation (to be completed by Theiagen developer)

• [x] Relevant documentation on the Public Health Resources "PHB Main" has been updated
• [x] Workflow diagrams have been updated to reflect changes. Not necessary

[kapsakcj]

@sage-wright FYI - I noticed that TheiaCoV_FASTA_Batch wasn't properly using organism_parameters.nextclade_dataset_tag output string as input for the final step in the workflow used for updating the sample-level data table.

Previously the workflow could only use the user-provided optional input nextclade_dataset_tag so if not provided by user, the value would be null and the sample-level data table would not be updated with the dataset tag that was actually used to run nextclade on the samples during the workflow

See last commit for the code update to resolve this.

Re-testing theiacov_fasta_batch here: https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/7e08d7ed-2e5a-42c1-b9b9-4a0ce63ad38c

⚠️ I need to review the outputs of this test ^

EDIT: ✅ Success! this commit resolved this issue. See the screenshot below of TSV files used for updating the sample-level data table before and after the code change:

[cimendes]

you are a ⭐