search

broadinstitute / cromwell

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments
http://cromwell.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
995 stars 360 forks source link

File evaluation in PAPI v2 #4140

Closed ruchim closed 6 years ago

ruchim commented 6 years ago

Based on a conversation with the Epigenomics group that is using Cromwell 34 on PAPI v2 -- it seems that a task can fail, not delocalize an expected output and the workflow will continue to start a downstream task but fail at running the command (as an expected input is missing). It's been confirmed that Cromwell was running in a Fail fast mode.

There seem to be two main requirements here:

  1. Cromwell should be evaluating if all expected outputs for a task exist before marking the task as a success.
  2. Cromwell should fail a job if it failed localize a specific input file.
  3. Cromwell should fail a job if it failed to delocalize any expected output file.

AC: Ensure that the PAPI v2 backend fulfills requirements #1-3.

cjllanwarne commented 6 years ago

I think there should be a #3: Cromwell should fail a job if it failed to delocalize a required output file.

ruchim commented 6 years ago

@mcovarr -- has this been fixed for PAPI v2?

mcovarr commented 6 years ago

There was a 34 hotfix for this a while back. This should also be fixed in Cromwells 35 and greater.