github-actions[bot]
commented
6 months ago Bleep bloop, I am a robot.
Alas, some of the Nextflow configuration tests failed!
test/configtest-F16.json
@ ["params","proc_resource_params","call_sIndel_Manta","retry_strategy","memory","operand"]
- "3 GB"
+ "5 GB"
@ ["params","proc_resource_params","call_sSNV_Mutect2","memory"]
- "3 GB"
+ "2 GB"
@ ["params","proc_resource_params","convert_BAM2Pileup_SAMtools","retry_strategy","memory","operand"]
- "3 GB"
+ "5 GB"
@ ["params","proc_resource_params","create_IndelCandidate_SAMtools","retry_strategy","memory","operand"]
- "3 GB"
+ "5 GB"
@ ["params","proc_resource_params","run_LearnReadOrientationModel_GATK","retry_strategy","memory","operand"]
- "2"
+ "20 GB"
@ ["params","proc_resource_params","run_LearnReadOrientationModel_GATK","retry_strategy","memory","strategy"]
- "exponential"
+ "add"
@ ["params","proc_resource_params","run_sump_MuSE","memory"]
- "24 GB"
+ "16 GB"
@ ["params","proc_resource_params","run_sump_MuSE","retry_strategy","memory","operand"]
- "8 GB"
+ "16 GB"
@ ["process","withName:call_sIndel_Manta","memory","2"]
- "9 GB"
+ "11 GB"
@ ["process","withName:call_sIndel_Manta","memory","3"]
- "12 GB"
+ "16 GB"
@ ["process","withName:call_sIndel_Manta","memory","closure"]
- "retry_updater(6 GB, add, 3 GB, $task.attempt, memory)"
+ "retry_updater(6 GB, add, 5 GB, $task.attempt, memory)"
@ ["process","withName:call_sSNV_Mutect2","memory","1"]
- "3 GB"
+ "2 GB"
@ ["process","withName:call_sSNV_Mutect2","memory","2"]
- "5 GB"
+ "4 GB"
@ ["process","withName:call_sSNV_Mutect2","memory","3"]
- "7 GB"
+ "6 GB"
@ ["process","withName:call_sSNV_Mutect2","memory","closure"]
- "retry_updater(3 GB, add, 2 GB, $task.attempt, memory)"
+ "retry_updater(2 GB, add, 2 GB, $task.attempt, memory)"
@ ["process","withName:convert_BAM2Pileup_SAMtools","memory","2"]
- "4 GB"
+ "6 GB"
@ ["process","withName:convert_BAM2Pileup_SAMtools","memory","3"]
- "7 GB"
+ "11 GB"
@ ["process","withName:convert_BAM2Pileup_SAMtools","memory","closure"]
- "retry_updater(1 GB, add, 3 GB, $task.attempt, memory)"
+ "retry_updater(1 GB, add, 5 GB, $task.attempt, memory)"
@ ["process","withName:create_IndelCandidate_SAMtools","memory","2"]
- "4 GB"
+ "6 GB"
@ ["process","withName:create_IndelCandidate_SAMtools","memory","3"]
- "7 GB"
+ "11 GB"
@ ["process","withName:create_IndelCandidate_SAMtools","memory","closure"]
- "retry_updater(1 GB, add, 3 GB, $task.attempt, memory)"
+ "retry_updater(1 GB, add, 5 GB, $task.attempt, memory)"
@ ["process","withName:run_LearnReadOrientationModel_GATK","memory","2"]
- "16 GB"
+ "28 GB"
@ ["process","withName:run_LearnReadOrientationModel_GATK","memory","closure"]
- "retry_updater(8 GB, exponential, 2, $task.attempt, memory)"
+ "retry_updater(8 GB, add, 20 GB, $task.attempt, memory)"
@ ["process","withName:run_sump_MuSE","memory","1"]
- "24 GB"
+ "16 GB"
@ ["process","withName:run_sump_MuSE","memory","closure"]
- "retry_updater(24 GB, add, 8 GB, $task.attempt, memory)"
+ "retry_updater(16 GB, add, 16 GB, $task.attempt, memory)"test/configtest-F32.json
@ ["params","proc_resource_params","call_sSNV_Mutect2","memory"]
- "3 GB"
+ "2 GB"
@ ["params","proc_resource_params","call_sSNV_Mutect2","retry_strategy","memory","operand"]
- "3 GB"
+ "2 GB"
@ ["params","proc_resource_params","run_LearnReadOrientationModel_GATK","memory"]
- "16 GB"
+ "12 GB"
@ ["params","proc_resource_params","run_LearnReadOrientationModel_GATK","retry_strategy","memory","operand"]
- "2"
+ "20 GB"
@ ["params","proc_resource_params","run_LearnReadOrientationModel_GATK","retry_strategy","memory","strategy"]
- "exponential"
+ "add"
@ ["params","proc_resource_params","run_sump_MuSE","memory"]
- "48 GB"
+ "18 GB"
@ ["process","withName:call_sSNV_Mutect2","memory","1"]
- "3 GB"
+ "2 GB"
@ ["process","withName:call_sSNV_Mutect2","memory","2"]
- "6 GB"
+ "4 GB"
@ ["process","withName:call_sSNV_Mutect2","memory","3"]
- "9 GB"
+ "6 GB"
@ ["process","withName:call_sSNV_Mutect2","memory","closure"]
- "retry_updater(3 GB, add, 3 GB, $task.attempt, memory)"
+ "retry_updater(2 GB, add, 2 GB, $task.attempt, memory)"
@ ["process","withName:run_LearnReadOrientationModel_GATK","memory","1"]
- "16 GB"
+ "12 GB"
@ ["process","withName:run_LearnReadOrientationModel_GATK","memory","3"]
- "64 GB"
+ "52 GB"
@ ["process","withName:run_LearnReadOrientationModel_GATK","memory","closure"]
- "retry_updater(16 GB, exponential, 2, $task.attempt, memory)"
+ "retry_updater(12 GB, add, 20 GB, $task.attempt, memory)"
@ ["process","withName:run_sump_MuSE","memory","1"]
- "48 GB"
+ "18 GB"
@ ["process","withName:run_sump_MuSE","memory","2"]
- "64 GB"
+ "34 GB"
@ ["process","withName:run_sump_MuSE","memory","3"]
- "64 GB"
+ "50 GB"
@ ["process","withName:run_sump_MuSE","memory","closure"]
- "retry_updater(48 GB, add, 16 GB, $task.attempt, memory)"
+ "retry_updater(18 GB, add, 16 GB, $task.attempt, memory)"If the above changes are surprising, stop and determine what happened.
If the above changes are expected, there are two ways to fix this:
- Automatically: Post a comment starting with "/fix-tests" (without the quotes) and I will update the tests for you (you must review my work afterwards).
- Manually: Follow these steps on Confluence.
sorelfitzgibbon
Description
Resource allocations updated based on results of 30 PCAWG and 13 SARC (SGSALASC) sarcoma sample runs. Main changes for F72:
convert_BAM2Pileup_SAMtoolsretry memory substantially increasedrun_sump_MuSEmemory allocation dropped from 80.GB to 18.GBPCAWG samples described here SARC samples described here
Sample coverage
Closes #255 #283
Testing Results
Tested on different PCAWG-63 samples that had failed before this update. They all failed again initially and worked on retry.
main logs:
/hot/code/sfitzgibbon/gitHub/uclahs-cds/pipeline-call-sSNV/sfitz-update-resources_DO220599_ssnv.log/hot/code/sfitzgibbon/gitHub/uclahs-cds/pipeline-call-sSNV/sfitz-update-resources_DO36881_ssnv.log/hot/code/sfitzgibbon/gitHub/uclahs-cds/pipeline-call-sSNV/sfitz-update-resources_DO37758_ssnv.log/hot/code/sfitzgibbon/gitHub/uclahs-cds/pipeline-call-sSNV/sfitz-update-resources_DO38847_ssnv.logprocess logs:
/hot/user/sfitzgibbon/PCAWG-63/data/pipeline-runs/sfitz-update-resources-DO220599-ssnv/call-sSNV-8.0.0/SA556937/log-call-sSNV-8.0.0-20240507T214647Z/hot/user/sfitzgibbon/PCAWG-63/data/pipeline-runs/sfitz-update-resources-DO36881-ssnv/call-sSNV-8.0.0/SA421337/log-call-sSNV-8.0.0-20240507T222604Z/hot/user/sfitzgibbon/PCAWG-63/data/pipeline-runs/sfitz-update-resources-DO37758-ssnv/call-sSNV-8.0.0/SA433958/log-call-sSNV-8.0.0-20240508T172838Z/hot/user/sfitzgibbon/PCAWG-63/data/pipeline-runs/sfitz-update-resources-DO38847-ssnv/call-sSNV-8.0.0/SA443714/log-call-sSNV-8.0.0-20240508T181628ZTesting after updating submodules,
nftestmain log:
/hot/software/pipeline/pipeline-call-sSNV/Nextflow/development/unreleased/sfitz-update-resources/slurm-154972.outprocess log:/hot/software/pipeline/pipeline-call-sSNV/Nextflow/development/unreleased/sfitz-update-resources/log-nftest-20240513T232120Z.logChecklist
[x] I have read the code review guidelines and the code review best practice on GitHub check-list.
[x] I have reviewed the Nextflow pipeline standards.
[x] The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].
[x] I have set up or verified the branch protection rule following the github standards before opening this pull request.
[x] I have added my name to the contributors listings in the
manifestblock in thenextflow.configas part of this pull request; I am listed already, or do not wish to be listed. (This acknowledgement is optional.)[x] I have added the changes included in this pull request to the
CHANGELOG.mdunder the next release version or unreleased, and updated the date.[x] I have updated the version number in the
metadata.yamlandmanifestblock of thenextflow.configfile following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)[x] I have tested the pipeline on at least one A-mini sample.