nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.76k stars 630 forks source link

Ability to use custom email template string instead of file #4583

Open stevekm opened 11 months ago

stevekm commented 11 months ago

The current implementation of custom email templates uses a separate template file, like this;

https://www.nextflow.io/docs/latest/config.html#scope-notification

notification {
    enabled = true
    template = "assets/mail/notification.txt"
}

Can we have this expanded so that we might be able to embed the contents of the template into the config file itself? Something like this


notification {
    enabled = true
    templateString = '''
====================================
 Workflow completion notification
====================================
 Run Name: ${workflow.runName}
<% if( workflow.success && workflow.stats.ignoredCount ) { %>
## Execution completed successfully. One or more tasks reported an error condition ##
<% } else if(workflow.success) { %>
## Execution completed successfully! ##
<% } else { %>
.... etc ...
'''
}

This way, we would be able to apply a custom notification from only the config file, with a command such as

nextflow -c /path/to/email.config run 'https://github.com/users/mypipeline'

instead of having to also worry about where to store the supplemental template file

( related to https://github.com/nextflow-io/nextflow/issues/4578 )

bentsherman commented 6 months ago

I think we will find a way to make the template file work better on the platform side. Many people have asked for CE-level config so it's a long time coming.

If you have many pipelines with the same common code, I wonder if it would be possible to create a Nextflow pipeline template, initialize a pipeline from the template, and also pull updates from that template in a streamlined manner?

I'm thinking of the nf-core modules command which allows you to install and update modules in your pipeline, and I think it does a git-style merge if there are conflicting changes. Do you think that kind of setup would work for you? You would still have to update each pipeline manually but the update process would be easier.

stevekm commented 6 months ago

well I am also looking at trying to address this at the Nextflow Plugin level as well

right now my thinking is that I can embed a base template (either a template-string or a template-creation method) in a Groovy Nextflow Plugin, expose some parameters for small customizations, and then use that within my pipelines.

bentsherman commented 6 months ago

I think that would also work

stevekm commented 1 month ago

just a follow-up on this, I did make some headway with custom Nextflow Plugin however I think I had previously filed some other tickets related to challenges on using it in prod in an enterprise setting, and I also did not ever get to a point to implement this exact feature yet. So I am wondering if there's any chance that a version of the feature as described in the original post might be feasible?