seqeralabs / nf-tower

Nextflow Tower system
https://tower.nf
Mozilla Public License 2.0
143 stars 51 forks source link

Tower not updated when sendMail() #311

Open daudn opened 3 years ago

daudn commented 3 years ago

I was working on integrating automatic email-ing of workflow output files, and then I noticed my dashboard wasn't being updated anymore (when I run a workflow, it doesn't show up in tower dashboard while executing). After completing, it then updates tower dashboard. Not a major issue, but possibly something that deserves to be pointed out. Below is steps to reproduce.

Normal working workflow (Runs + updates in tower dashboard):

nextflow run text.nf -c nextflow.config

process create_contents {
    executor 'local'

    output:
    set file("first_file.txt"), file("second_file.csv"), file("all_amplicon_analysis.txt") into email_output

    script:
    """
    touch first_file.txt, second_file.csv, all_amplicon_analysis.txt
    echo some_content > first_file.txt
    echo A,B > second_file.csv
    echo fasta > all_amplicon_analysis.txt
    """
}

Does not update in tower dashboard:

process create_contents {

    executor 'local'

    output:
    set file("first_file.txt"), file("second_file.csv"), file("all_amplicon_analysis.txt") into email_output

    script:
    """
    touch first_file.txt, second_file.csv, all_amplicon_analysis.txt
    echo some_content > first_file.txt
    echo A,B > second_file.csv
    echo fasta > all_amplicon_analysis.txt
    """
}

def mail_subject = "mail_subject"

sendMail {
        to '***'
        from '***'
        subject mail_subject
        '''
        Pipeline execution summary
        '''
    }

This does not update in Tower Dashboard.

This is my nextflow.config

cleanup = true

process {
    executor = 'google-lifesciences'
}

cloud {
    driver = 'google'
}

google {
    project = "***"
    region = "***"
    lifeSciences.bootDiskSize = 20.GB
    lifeSciences.preemptible = true
}

docker {
    enabled = true
}

tower {
  accessToken = "***"
  enabled = true
}

mail {
    smtp.host = 'smtp.office365.com'
    smtp.port = 587
    smtp.user = "***"
    smtp.password = "***"
    smtp.auth = true
    smtp.starttls.enable = true
    smtp.starttls.required = true
    smtp.ssl.trust = 'smtp.office365.com'
    smtp.ssl.protocols = 'TLSv1.2'

}
pditommaso commented 3 years ago

What do you mean by does not update ?

daudn commented 3 years ago

I mean there is no real-time monitoring of the workflow. Once the workflow completes/fails, it then updates all-at-once at the end. If I remove sendMail(), it works as usual (real time updates as the workflow executes)

pditommaso commented 3 years ago

Ok, got it, thanks for reporting. Think the main problem is that sendMail is not supported at this time when launching from Tower. we'll investigate the other problem.

daudn commented 3 years ago

This seems to be resolved! Getting live workflow updates with sendMail() included.

Thank you @pditommaso