[Internal] Updating runtime parameters

[sage-wright]

This one is a doozy.

This PR:

• closes #483
• closes #485
• closes #487
• closes #488
• closes #489
• closes #490
• closes #491
• closes #492
• closes #493

🗑️ This dev branch should be deleted after merging to main.

:brain: Aim, Context and Functionality

Many tasks require far more runtime parameters than are actually used. This PR adjusts various runtime parameters accordingly.

:hammer_and_wrench: Impacted Workflows/Tasks & Changes Being Made

This will affect the behavior of the workflow(s) even if users don’t change any workflow inputs relative to the last version : No

Running this workflow on different occasions could result in different results, e.g. due to use of a live database, "latest" docker image, or stochastic data processing : No

• No change in workflow function, just optimization of resources

:clipboard: Workflow/Task Step Changes

🔄 Data Processing

Compute resources changed:

• many, see also code changes

➡️ Inputs

None

⬅️ Outputs

None

:test_tube: Testing

Terra Testing

Suggested Scenarios for Reviewer to Test

Ensure costs and runtime attributes went down as expected.

Theiagen Version Release Testing (optional)

None

:microscope: Final Developer Checklist

• [x] The workflow/task has been tested locally and results, including file contents, are as anticipated
• [ ] The workflow/task has been tested on Terra and results, including file contents, are as anticipated
• [x] The CI/CD has been adjusted and tests are passing (to be completed by Theiagen developer)
• [x] Code changes follow the style guide

🎯 Reviewer Checklist

• [x] All impacted workflows/tasks have been tested on Terra with a different dataset than used for development
• [x] All reviewer-suggested scenarios have been tested and any additional
• [x] All changed results have been confirmed to be accurate
• [x] All workflows/tasks impacted by change/s have been tested using a standard validation dataset to ensure no unintended change of functionality
• [x] All code adheres to the style guide
• [x] MD5 sums have been updated
• [x] The PR author has addressed all comments

🗂️ Associated Documentation (to be completed by Theiagen developer)

• [ ] Relevant documentation on the Public Health Resources "PHB Main" has been updated
• [ ] Workflow diagrams have been updated to reflect changes

[AndrewLangvt]

TheiaProk_Illumina_PE_PHB - https://app.terra.bio/#workspaces/theiagen-training-workspaces/Theiagen_Lang_Sandbox/job_history/4fa24d88-2417-4fa3-a3e1-3f4268ca1198 shovil - confirmed CPUs are passed to shovil command busco - confirmed preemtible gambit - confirmed disk size 20, confirmed mem 2, CPU 1

realized I had to pass in flag to run midas- https://app.terra.bio/#workspaces/theiagen-training-workspaces/Theiagen_Lang_Sandbox/job_history/1fa8a187-17f8-4638-9ba3-965adb8e1c73 midas - confirmed preemtible, mem = 4

@jrotieno from my perspective, once you review your set of these workflows, you can mark it approved for merge

[jrotieno]

For the additional TheiaProk tasks assigned to me, I just used Andrew Lang's runs: check-reads: 2GB RAM, 100GB Disk, 1 CPU and preemptible TRUE cg-pipeline: 2GB RAM, 50GB Disk, 1 CPU and preemptible TRUE version_capture container used is the alpine-plus-bash. Not seen the version though (3.20.0), should I have in the compute details? 1GB RAM, 10GB Disk, 1 CPU and preemptible TRUE

kSNP3 CPUs: https://job-manager.dsde-prod.broadinstitute.org/jobs/b7e86f67-8764-49d4-8456-b0d3d7f7944a 4GB RAM, 100GB Disk, 2 CPU and preemptible FALSE

[jrotieno]

@AndrewLangvt maybe you just need to mark the ones you went through as done on the board?

[AndrewLangvt]

@jrotieno good call; done!