broadinstitute / cromwell

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments
http://cromwell.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
1k stars 360 forks source link

Saving mistakes in Cromwell logs #2210

Open antonkulaga opened 7 years ago

antonkulaga commented 7 years ago

Whenever I run workflow with mistake a see some weird errors in cromwell-server akka-logs while nothing is written to cromwell-workflow-logs Here is a simple example:

workflow worms {

  File samplesFile

  #Name \t File
  Array[Array[File]] samples = read_tsv(samplesFile)

  scatter (sample in samples) {
    call stats {
        input:
            fileName = sample(0), #simple mistake with "()" instead of "[]"
            file =  sample(1)
    }
  }
}

task stats {

  String fileName
  File file

  command {
    /opt/sratoolkit/sra-stat ${file} > stats.txt
  }

  runtime {
    docker: "itsjeffreyy/sratoolkit"
  }

  output {
    File stats = "stats.txt"
  }
}

Here I make quite typical mistake by using () instead of []. But when I send it to the server for execution it I get an empty cromwell-workflow-logs folder and errors are displayed only in stdout of the cromwell server. What I expect is to see them in cromwell-workflow-logs (if my expectations are wrong, it would be nice to have in documentation a description of which log folder are for which type of errors) By the way, according to the error I get, you use runtime reflection to search for "sample" function (that is perceived as a function instead of array due to round brackets). Not the safest, way to identify functions, IMHO.

katevoss commented 7 years ago

@geoffjentry do you have any thoughts on this?