Update womtool graph command to support WDL 1.0

[pshapiro4broad]

Now that we've updated our WDLs to 1.0, we've found that womtool graph no longer works. It looks like it only supports draft2 and earlier WDL.

test.wdl:

version 1.0

workflow Test { }

$ java -jar womtool-35.jar graph /tmp/test.wdl 
Exception in thread "main" wdl.draft2.parser.WdlParser$SyntaxError: ERROR: Finished parsing without consuming all tokens.

version 1.0
^

    at wdl.draft2.parser.WdlParser.parse(WdlParser.java:2330)
    at wdl.draft2.parser.WdlParser.parse(WdlParser.java:2335)
    at wdl.draft2.model.AstTools$.getAst(AstTools.scala:266)
    at wdl.draft2.model.WdlNamespace$.$anonfun$load$1(WdlNamespace.scala:160)
    at scala.util.Try$.apply(Try.scala:209)
    at wdl.draft2.model.WdlNamespace$.load(WdlNamespace.scala:160)
    at wdl.draft2.model.WdlNamespace$.loadUsingSource(WdlNamespace.scala:156)
    at wdl.draft2.model.WdlNamespaceWithWorkflow$.load(WdlNamespace.scala:571)
    at womtool.graph.GraphPrint$.generateWorkflowDigraph(GraphPrint.scala:19)
    at womtool.WomtoolMain$.graph(WomtoolMain.scala:94)
    at womtool.WomtoolMain$.dispatchCommand(WomtoolMain.scala:48)
    at womtool.WomtoolMain$.runWomtool(WomtoolMain.scala:125)
    at womtool.WomtoolMain$.delayedEndpoint$womtool$WomtoolMain$1(WomtoolMain.scala:130)
    at womtool.WomtoolMain$delayedInit$body.apply(WomtoolMain.scala:18)
    at scala.Function0.apply$mcV$sp(Function0.scala:34)
    at scala.Function0.apply$mcV$sp$(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App.$anonfun$main$1$adapted(App.scala:76)
    at scala.collection.immutable.List.foreach(List.scala:389)
    at scala.App.main(App.scala:76)
    at scala.App.main$(App.scala:74)
    at womtool.WomtoolMain$.main(WomtoolMain.scala:18)
    at womtool.WomtoolMain.main(WomtoolMain.scala)

The womgraph command still works, but the output from that command is so verbose it's unusable for viewing our workflows.

[aednichols]

This is an opportunity to consider making graph operate on WOM and therefore automatically work on any language and language version

[hailiangmei]

Is there any news on this bug fixing? Or is there an alternative to create graph for WDL 1.0 files? Thanks!

[cjllanwarne]

This is actually partially implemented in https://github.com/broadinstitute/cromwell/pull/4522 - perhaps it would make sense to separate out the "DAG improvement" from the "live DAG view"

[hailiangmei]

Would that be possible to increase the priority of this issue? Would be really nice to easily generate the workflow graph for discussion and presentation purpose. Thanks!

[aednichols]

@cjllanwarne

[tsjsdbd]

We also need this feature, any news fot this ? (it's been a year :smile: )

[huboqiang]

We also need this too.

[vdauwera]

@cjllanwarne AAAAAAAAAAH I really need this for the book, what needs to happen for this to get prioritized?

[mes5k]

By temporarily deleting version 1.0, removing input and associated brace characters, and changing and ~{whatever} to ${whatever} I'm still able to generate graphs. It's irritating, but it'll work in a pinch.

[hailiangmei]

@mes5k thanks for the tip! In our biowdl project, we have imported sub workflows and imported tasks, so such manual hacks by a user would be quite painful. But this could be a good hint to have a short-term workaround solution in womtool?

[aednichols]

Have y'all tried womtool womgraph?

~My understanding is that womgraph is designed to be the successor to graph. This is similar to how wdltool became womtool. We have failed to communicate this migration adequately.~ Not fully correct as it turns out; but womgraph may fill a need until graph is updated.

[vdauwera]

Note for the others (since @aednichols and I are discussing this offline) -- womgraph includes inputs/outputs in the graph, which graph does not. My $0.02 is that this can be either a very good thing or a very bad thing depending on the complexity of your pipeline: for a simple one it' really nice, but for a very complex one it makes it really hard to read.

[mes5k]

womgraph produced vastly too much output for my (apparently) complex pipeline. dot couldn't render it in png at all and produced a corrupted pdf file. I think that an option that produces "simple" output that non-developers can look at and mostly comprehend, like the old graph option, is still very desirable.

[vdauwera]

I agree, would be great to be able to suppress the extra information, perhaps with a flag

[pshapiro4broad]

Have y'all tried womtool womgraph?

As I said above:

The womgraph command still works, but the output from that command is so verbose it's unusable for viewing our workflows.

womgraph may be helpful for simple workflows or debugging womtool but it's not a substitute for graph when viewing more complex workflows.

For example, here's the output for the GATK best practices exome pipeline.

exome.pdf

[vdauwera]

Good news for everyone who's been waiting on this: @cjllanwarne has a PR that fixes this and improves the visualization of conditionals and imports. See https://github.com/broadinstitute/cromwell/pull/5326