This PR introduce the exomiser tool in the pipeline. Additionally, it introduces a new parameter called tools, which allows users to specify whether VEP, Exomiser, or both should be executed.
Additionally, there is some refactoring in the main workflow (postprocessing.nf) to use the def keyword and more standard variable names. I could not do it only for exomiser variables as, strangely, the code was not compiling when a channel with local scope was derived from a channel with non-local scope.
For the schema, if-else blocks are used to conditionally require exomiser/vep parameters based on the value of the tools parameter.
If an exomiser or vep parameter is removed/added/modified, the logic in the if-else block will need to be manually updated. For example, if the removed parameter was required, it must be manually removed from the list of required parameters.
Tests
Running the pipeline:
Run the pipeline locally with the public test dataset (V3).
Check that vep output files are under results/vep and exomiser output files under results/exomiser/results.
Check that vep and exomiser output files are non empty.
Run the pipeline locally with the public test dataset (V3) again, this time commenting in the test config parameters exomiser_analysis_wes and exomiser_analysis_wgs. We want to see if the pipeline is correctly executed using the default analysis files.
Run the pipeline on juno with the public test dataset (V3).
Make sure that ci tests pass
Note: the V3 dataset covers REMM and CADD data sources and produces non empty exomiser files.
Linter:
Run the linter locally and compare with the main branch
Test the schema:
Check that the nf-core schema build commands still works and does not break the added if-else blocks.
nf-core schema lintnf-core schema docsnf-core schema validate . data-test/paramsJuno.jsonnf-core schema build
The validate command seem to take into account if-else blocks correctly according to my tests.
I tweaked the paramsJuno.json file to remove a required exomiser parameter and it reported it correctly.
Test exomiser/vep parameter validation (stub mode)
exomiser tool present:
missing required parameter: should be detected and pipeline should stop
missing optional parameter (ex: cadd version): pipeline should run with exomiser
remm/cadd version specified:
missing other remm/cadd parameters: missing parameters should be detected and pipeline should stop
all remm/cadd paremeters specified: pipeline should run exomiser and pass remm/cadd options
remm/cadd version not specified:
missing other remm/cadd parameters: pipeline should run exomiser and not pass remm/cadd options
exomiser_analysis_wes / exomiser_analysis_wgs not specified: pipeline should use the default analysis files
exomiser tool absent:
missing required/optional exomiser parameter: pipeline should run without exomiser
remm/cadd versions specified:
missing other remm/cadd parameters: pipeline should run without exomiser
vep tool present:
missing required vep parameter: should be detected and pipeline should stop
vep tool absent:
missing required/optional vep parameters: pipeline should run without vep
check that exomiser is run if tools is set to vep,exomiser | exomiser,vep | exomiser
check that exomiser is skipped if tools is set to vep | empty string | null
check that vep is run if tools is set to vep,exomiser | exomiser,vep | vep
check that vep is skipped if tools is set to exomiser | empty string | null
Extra tests with CADD/REMM
If cadd/remm parameters are not passed (i.e. missing cadd/remm version) and the analysis file includes cadd/remm pathogenicity sources, the pipeline should be executed, but it should fail at the exomiser step
if cadd/remm parameters are passed and the analysis file does not include cadd/remm pathogenicity sources, the pipeline should be executed successfully
Validate json format is supported for analysis files and familyPheno files:
ask copilot to generate a json copy of one of the default analysis file and pass it at the command line
ask copilot to generate a json copy of one of the family file, create a copy of the test sample sheet pointing on the json family file and run the pipeline locally with it
PR checklist
[x] This comment contains a description of changes (with reason).
[x] If you've fixed a bug or added code that should be tested, add tests!
[x] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[x] Make sure your code lints (nf-core lint).
[x] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
[ ] Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
[x] Usage Documentation in docs/usage.md is updated.
[x] Output Documentation in docs/output.md is updated.
[x] CHANGELOG.md is updated.
[x] README.md is updated (including new tool citations and authors/contributors).
This PR introduce the exomiser tool in the pipeline. Additionally, it introduces a new parameter called
tools, which allows users to specify whether VEP, Exomiser, or both should be executed.Additionally, there is some refactoring in the main workflow (
postprocessing.nf) to use thedefkeyword and more standard variable names. I could not do it only for exomiser variables as, strangely, the code was not compiling when a channel with local scope was derived from a channel with non-local scope.Here is the associated JIRA for exomiser: https://ferlab-crsj.atlassian.net/browse/CLIN-2947
Limitations
For the schema, if-else blocks are used to conditionally require exomiser/vep parameters based on the value of the
toolsparameter.If an exomiser or vep parameter is removed/added/modified, the logic in the if-else block will need to be manually updated. For example, if the removed parameter was required, it must be manually removed from the list of required parameters.
Tests
Running the pipeline:
Note: the V3 dataset covers REMM and CADD data sources and produces non empty exomiser files.
Linter:
Test the schema:
Check that the nf-core schema build commands still works and does not break the added if-else blocks.
nf-core schema lintnf-core schema docsnf-core schema validate . data-test/paramsJuno.jsonnf-core schema buildTest exomiser/vep parameter validation (stub mode)
Extra tests with CADD/REMM
Validate json format is supported for analysis files and familyPheno files:
PR checklist
nf-core lint).nextflow run . -profile test,docker --outdir <OUTDIR>).nextflow run . -profile debug,test,docker --outdir <OUTDIR>).docs/usage.mdis updated.docs/output.mdis updated.CHANGELOG.mdis updated.README.mdis updated (including new tool citations and authors/contributors).