nf-core / eager

A fully reproducible and state-of-the-art ancient DNA analysis pipeline
https://nf-co.re/eager
MIT License
148 stars 82 forks source link

Only load modules if being used #830

Open jfy133 opened 2 years ago

jfy133 commented 2 years ago

I realised that in DSL2 there is possibly one way to reduce the size of the console logging of all the processes (which is currently very large for eager, and ends up a user losing any warnings at the top of the very long list of processes).

We could maybe not have these displayed if we only include those modules when they are requested.

e.g.

if ( params.run_deeparg ) {
    include { DEEPARG_DOWNLOADDATA } from '../modules/nf-core/modules/deeparg/downloaddata/main'
    include { DEEPARG_PREDICT      } from '../modules/nf-core/modules/deeparg/predict/main'
}

Note: each invocation of those modules will also need to be wrapped in params.run_<tool>

if ( params.run_deeparg ) {
    DEEPARG_PREDICT ( ch_input_for_deeparg, ch_deeparg_db )
    ch_versions = ch_versions.mix(DEEPARG_PREDICT.out.versions)
}

Current issue: get a

WARN: There's no process matching config selector: DEEPARG_PREDICT

But maybe this can also be invoked conditionally within modules.config?

And otherwise, DEEPARG is thus not displayed unless --run_deeparg is supplied

[a6/ea47cb] process > NFCORE_FUNCSCAN:FUNCSCAN:INPUT_CHECK:SAMPLESHEET_CHECK (input.csv) [100%] 1 of 1 ✔
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:GUNZIP                                    -
[f5/f68fbf] process > NFCORE_FUNCSCAN:FUNCSCAN:PROKKA (deeparg1)                         [100%] 1 of 1, failed: 1 ✘
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:FARGENE                                   -
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:CUSTOM_DUMPSOFTWAREVERSIONS               -
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:MULTIQC                                   [  0%] 0 of 1
TCLamnidis commented 7 months ago

The issue with the config selector warning is something that has been "fixed" by the conventions we implemented in dev_docs.md. The point about making the ansi log shorter is still valid, though not that important i guess.