search

ngless-toolkit / ngless

NGLess: NGS with less work
https://ngless.embl.de
Other
142 stars 24 forks source link

CWL tool descriptions are missing outputs, therefore not usable in CWL workflows #26

Closed mr-c closed 7 years ago

mr-c commented 7 years ago

Hey @luispedro , thanks for using the argparse2tool to generate CWL descriptions in https://github.com/luispedro/ngless/tree/master/scripts

However there are no output stanzas as argparse models the inputs to a program, not the outputs.

While the CWL ngless descriptions can be used to run a ngless command standalone, they can not be used in CWL workflows as stated in https://ngless.readthedocs.io/en/latest/faq.html#what-is-the-relationship-of-ngless-to-the-common-workflow-language due to the lack of outputs

At https://github.com/erasche/argparse2tool#cwl-specific-functionality we document how to pass in hand-written outputs stanzas using --output_section when invoking argparse2tool.

Alternatively I see some JSON-esque code at https://github.com/luispedro/ngless/blob/master/scripts/ngless-count.py#L35

You can pass the CWL input object directly as JSON to any tool and skip all the argument parsing complexity. Likewise any CWL compliant platform is able to consume JSON from a tool to learn at run time the actual outputs and their locations + any optional metadata.

I think DSLs are pretty cool and useful and I'd love to see more that compile or convert to CWL giving everyone the best of all worlds!

luispedro commented 7 years ago

Hi @mr-c, thanks for the very detailed issue explanation.

/cc @Unode : thoughts on how to address this?

unode commented 7 years ago

@mr-c indeed, thanks for all the information.

With regards to the JSON-esque code, this was an accidental coincidence. I'm not sure I could use this directly for CWL specification.

I need to digest the issue a bit longer. outputs are indeed a problem but for the limited subset of functions we should be able to know what outputs we are going to get. Not sure how far we can automate this though.

Another point is that we'll need tests to ensure we don't break the scripts if some function API changes.

luispedro commented 7 years ago

Your combined efforts on #27 have solved this issue here!