search

nf-core / fastquorum

Pipeline to produce consensus reads using unique molecular indexes/barcodes (UMIs)
https://nf-co.re/fastquorum
MIT License
20 stars 9 forks source link

Add when conditions to prepare #58

Closed SPPearce closed 3 months ago

SPPearce commented 5 months ago

As noted in #56 , the BWA index generation happens every time (and doesn't seem to cache, which makes it even more annoying). This PR adds when conditions to the prepare genome steps so they don't run if the appropriate parameters are passed.

github-actions[bot] commented 5 months ago

nf-core lint overall result: Passed :white_check_mark: :warning:

Posted for pipeline commit 916a1bf

+| ✅ 169 tests passed       |+
#| ❔   8 tests were ignored |#
!| ❗   4 tests had warnings |!
### :heavy_exclamation_mark: Test warnings: * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config ``manifest.version`` should end in ``dev``: ``1.0.0`` * [readme](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/readme) - README contains the placeholder `zenodo.XXXXXXX`. This should be replaced with the zenodo doi (after the first release). * [pipeline_todos](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/pipeline_todos) - TODO string in `README.md`: _Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file._ * [pipeline_todos](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/pipeline_todos) - TODO string in `output.md`: _Write this documentation describing your workflow's output_ ### :grey_question: Tests ignored: * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File is ignored: `.github/workflows/awsfulltest.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File is ignored: `.github/workflows/awstest.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File is ignored: `conf/modules/modules.config` * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - File ignored due to lint config: `.github/PULL_REQUEST_TEMPLATE.md` * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - File ignored due to lint config: `assets/nf-core-fastquorum_logo_light.png` * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - File ignored due to lint config: `.gitignore` or `.prettierignore` * [actions_ci](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_ci) - actions_ci * [template_strings](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/template_strings) - template_strings ### :white_check_mark: Tests passed: * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.gitattributes` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.gitignore` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.nf-core.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.editorconfig` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.prettierignore` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.prettierrc.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `CHANGELOG.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `CITATIONS.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `CODE_OF_CONDUCT.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `LICENSE` or `LICENSE.md` or `LICENCE` or `LICENCE.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `nextflow_schema.json` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `nextflow.config` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `README.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/.dockstore.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/CONTRIBUTING.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/ISSUE_TEMPLATE/bug_report.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/ISSUE_TEMPLATE/config.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/ISSUE_TEMPLATE/feature_request.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/PULL_REQUEST_TEMPLATE.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/workflows/branch.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/workflows/ci.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/workflows/linting_comment.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `.github/workflows/linting.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `assets/email_template.html` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `assets/email_template.txt` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `assets/sendmail_template.txt` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `assets/nf-core-fastquorum_logo_light.png` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `conf/modules.config` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `conf/test.config` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `conf/test_full.config` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `docs/images/nf-core-fastquorum_logo_light.png` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `docs/images/nf-core-fastquorum_logo_dark.png` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `docs/output.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `docs/README.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `docs/README.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `docs/usage.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `main.nf` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `assets/multiqc_config.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `conf/base.config` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `conf/igenomes.config` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File found: `modules.json` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `.github/ISSUE_TEMPLATE/bug_report.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `.github/ISSUE_TEMPLATE/feature_request.md` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `.github/workflows/push_dockerhub.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `.markdownlint.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `.nf-core.yaml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `.yamllint.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `bin/markdown_to_html.r` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `conf/aws.config` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `docs/images/nf-core-fastquorum_logo.png` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/Checks.groovy` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/Completion.groovy` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/NfcoreTemplate.groovy` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/Utils.groovy` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/Workflow.groovy` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/WorkflowMain.groovy` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/WorkflowFastquorum.groovy` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `parameters.settings.json` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `pipeline_template.yml` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `Singularity` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `lib/nfcore_external_java_deps.jar` * [files_exist](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_exist) - File not found check: `.travis.yml` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.name` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.nextflowVersion` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.description` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.version` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.homePage` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `timeline.enabled` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `trace.enabled` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `report.enabled` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `dag.enabled` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `process.cpus` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `process.memory` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `process.time` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `params.outdir` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `params.input` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `params.validationShowHiddenParams` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `params.validationSchemaIgnoreParams` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `manifest.mainScript` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `timeline.file` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `trace.file` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `report.file` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable found: `dag.file` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.nf_required_version` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.container` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.singleEnd` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.igenomesIgnore` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.name` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable (correctly) not found: `params.enable_conda` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config ``timeline.enabled`` had correct value: ``true`` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config ``report.enabled`` had correct value: ``true`` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config ``trace.enabled`` had correct value: ``true`` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config ``dag.enabled`` had correct value: ``true`` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config ``manifest.name`` began with ``nf-core/`` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable ``manifest.homePage`` began with https://github.com/nf-core/ * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config ``dag.file`` ended with ``.html`` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config variable ``manifest.nextflowVersion`` started with >= or !>= * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config `params.custom_config_version` is set to `master` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config `params.custom_config_base` is set to `https://raw.githubusercontent.com/nf-core/configs/master` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Lines for loading custom profiles found * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - nextflow.config contains configuration profile `test` * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.mode= rd * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.custom_config_version= master * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.max_cpus= 16 * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.max_memory= 128.GB * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.max_time= 240.h * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.publish_dir_mode= copy * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.max_multiqc_email_size= 25.MB * [nextflow_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nextflow_config) - Config default value correct: params.validate_params= true * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.gitattributes` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.prettierrc.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `CODE_OF_CONDUCT.md` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `LICENSE` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/.dockstore.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/CONTRIBUTING.md` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/ISSUE_TEMPLATE/bug_report.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/ISSUE_TEMPLATE/config.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/ISSUE_TEMPLATE/feature_request.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/workflows/branch.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/workflows/linting_comment.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `.github/workflows/linting.yml` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `assets/email_template.html` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `assets/email_template.txt` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `assets/sendmail_template.txt` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `docs/images/nf-core-fastquorum_logo_light.png` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `docs/images/nf-core-fastquorum_logo_dark.png` matches the template * [files_unchanged](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/files_unchanged) - `docs/README.md` matches the template * [actions_awstest](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_awstest) - '.github/workflows/awstest.yml' is triggered correctly * [actions_awsfulltest](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_awsfulltest) - `.github/workflows/awsfulltest.yml` is triggered correctly * [actions_awsfulltest](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_awsfulltest) - `.github/workflows/awsfulltest.yml` does not use `-profile test` * [readme](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/readme) - README Nextflow minimum version badge matched config. Badge: `23.04.0`, Config: `23.04.0` * [pipeline_name_conventions](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/pipeline_name_conventions) - Name adheres to nf-core convention * [schema_lint](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/schema_lint) - Schema lint passed * [schema_lint](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/schema_lint) - Schema title + description lint passed * [schema_lint](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/schema_lint) - Input mimetype lint passed: 'text/csv' * [schema_params](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/schema_params) - Schema matched params returned from nextflow config * [system_exit](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/system_exit) - No `System.exit` calls found * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: branch.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: ci.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: awsfulltest.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: fix-linting.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: linting.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: download_pipeline.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: release-announcements.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: clean-up.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: awstest.yml * [actions_schema_validation](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/actions_schema_validation) - Workflow validation passed: linting_comment.yml * [merge_markers](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/merge_markers) - No merge markers found in pipeline files * [modules_json](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_json) - Only installed modules found in `modules.json` * [multiqc_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/multiqc_config) - `assets/multiqc_config.yml` found and not ignored. * [multiqc_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/multiqc_config) - `assets/multiqc_config.yml` contains `report_section_order` * [multiqc_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/multiqc_config) - `assets/multiqc_config.yml` contains `export_plots` * [multiqc_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/multiqc_config) - `assets/multiqc_config.yml` contains `report_comment` * [multiqc_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/multiqc_config) - `assets/multiqc_config.yml` follows the ordering scheme of the minimally required plugins. * [multiqc_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/multiqc_config) - `assets/multiqc_config.yml` contains a matching 'report_comment'. * [multiqc_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/multiqc_config) - `assets/multiqc_config.yml` contains 'export_plots: true'. * [modules_structure](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_structure) - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL' * [base_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/base_config) - `conf/base.config` found and not ignored. * [base_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/base_config) - `FASTQC` found in `conf/base.config` and Nextflow scripts. * [base_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/base_config) - `MULTIQC` found in `conf/base.config` and Nextflow scripts. * [modules_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_config) - `conf/modules.config` found and not ignored. * [modules_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_config) - `BWA_INDEX` found in `conf/modules.config` and Nextflow scripts. * [modules_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_config) - `SAMTOOLS_FAIDX` found in `conf/modules.config` and Nextflow scripts. * [modules_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_config) - `SAMTOOLS_DICT` found in `conf/modules.config` and Nextflow scripts. * [modules_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_config) - `FASTQC` found in `conf/modules.config` and Nextflow scripts. * [modules_config](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/modules_config) - `MULTIQC` found in `conf/modules.config` and Nextflow scripts. * [nfcore_yml](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nfcore_yml) - Repository type in `.nf-core.yml` is valid: `pipeline` * [nfcore_yml](https://nf-co.re/tools/docs/2.14.1/pipeline_lint_tests/nfcore_yml) - nf-core version in `.nf-core.yml` is set to the latest version: `2.14.1` ### Run details * nf-core/tools version 2.14.1 * Run at `2024-06-05 14:57:22`
nh13 commented 3 months ago

@znorgaard can you verify that the bwa index process is skipped when (1) run a second time, and (2) manually copied and run for a first time?

znorgaard commented 3 months ago

@nh13, This seems to be working as expected.

The goal of this PR seems to be to skip generation of the files if they're explicitly provided as parameters. I don't think we're expecting it to skip generation of the files if the files are found to exist in the same directory as the reference.

Example run

I'm using Nextflow version 24.04.2, with this branch (prepare_genome_when).

OUTDIR="/full/path/to/output"
nextflow run main.nf \
    -profile test,docker \
    --outdir ${OUTDIR} \
    --save_reference

Completes successfully and includes the reference index files and dict.

Example resume run

nextflow run main.nf \
    -profile test,docker \
    -resume \
    --outdir ${OUTDIR} \
    --save_reference

Completes successfully and everything except MULTIQC is pulled from cache.

Example run with specified outputs

nextflow run main.nf \
    -profile test,docker \
    --outdir ${OUTDIR} \
    --bwa ${OUTDIR}/genome/bwa/ \
    --dict ${OUTDIR}/genome/chr17.fa.dict \
    --fasta_fai ${OUTDIR}/genome/chr17.fa.fai

Skips creation of the reference index files and dict using the provided paths instead.

If you move the files and provide updated paths it also behaves as expected, still skipping the creation of those resources.

mv ${OUTDIR}/genome ${OUTDIR}/my-custom-genome
nextflow run main.nf \
    -profile test,docker \
    --outdir ${OUTDIR} \
    --bwa ${OUTDIR}/my-custom-genome/bwa/ \
    --dict ${OUTDIR}/my-custom-genome/chr17.fa.dict \
    --fasta_fai ${OUTDIR}/my-custom-genome/chr17.fa.fai

If this doesn't cover the specific test case you're thinking of, I'm happy to run more.

SPPearce commented 3 months ago

Yes, this will only check if the command line arguments are given, and seems to be working at that.