PMCC-BioinformaticsCore / janis-core

Core python modules for Janis Pipeline workflow assistant
GNU General Public License v3.0
4 stars 9 forks source link

Refactor codetool to use JSON input #12

Open illusional opened 4 years ago

illusional commented 4 years ago

Edit 2021-01-20: I rewrote this description to be clearer, and updated it to reflect latest information.

Sometimes it's difficult to get specific types of params into a code tool, restricted by the CLI interface, both arg-parse and the CWL / WDL command line generators, so writing to a JSON file, then just parsing that in should make it a lot easier. CWL worked pretty much out of the box, but generating the JSON file is a little bit tricker in Cromwell due to some grammar issues.

Minimal example

version development
task echo_writestring {
  input {
    Int name = "Hello!"
  }
  command <<<
    cat ~{write_json({"name": name})}
  >>>
  output {
    String out = read_string(stdout())
  }
}

This gives the following error for Cromwell:

ERROR: Unexpected symbol (line 9, col 36) when parsing '_gen26'.
Expected rbrace, got "}".
    echo ~{write_json({"name": name})}
                                   ^
$e = :identifier <=> :lparen $_gen24 :rparen -> FunctionCall( name=$0, params=$2 )

There was talk about cromwell upgrading the grammar which I thought might fix this, but it's been sitting in the backlog for a while: JIRA: CROM-6280, Cromwell Slack.

Next, I tried inserting an input so the write_json wouldn't be inside a string interpoloated block which got me a lot closer.

Error: No suitable input for 'computed' amongst '{intvalue}' Jira issue: https://broadworkbench.atlassian.net/browse/BW-392

task echo_writestring {
  input {
    Int intvalue = 1
    Int computed = intvalue + 1
  }
  command <<<
    echo ~{computed}
  >>>
  output {
    String out = read_string(stdout())
  }
}

But interestingly this turns out to work if wrapped in a workflow. So merging this PR actually would work, UNLESS you wanted to directly run a single python tool without wrapping it in a workflow. As of 20th Jan, there was a bit of movement on that Cromwell Jira ticket for the computed values, so if that's resolved, this PR should be good to go!