Annotating WDL

[joshua-gould]

My need for annotations is to auto-generate a user interface similar to Galaxy (https://docs.galaxyproject.org/en/master/dev/schema.html). Annotations would allow FireCloud or another client to automatically generate the user interface for the WDL (e.g. form fields, text, help, etc.). For example: ` workflow myWorkflow{

@ui {"title":"my input", "options":['a','b,'c']}

String s

@ui {"title":"my checkbox", "description":"xxx"}

Boolean? b

@ui {"title":"bar"}

Int? i 

`

would generate a form similar to:

Annotations probably have other use cases that could help others.

[orodeh]

I agree with the need, at DNAnexus we also want to be to annotate declarations. An example use case is for adding help strings to task/workflow inputs.

[cjllanwarne]

Is it possible to try to include these in the existing parameter_meta section? TBH I'm reluctant to add UI specific annotations to the main body but typed enumerations are often requested and more structure in parameter_meta seems sensible too.

[orodeh]

I believe the things we need could be satisfied by extending the parameter_meta section.

The main missing piece for DNAnexus is extra information for each task/workflow input (and output field). These are the main ones we would like to support this way:

• Help: a string with documentation for a task/workflow parameter
• Patterns: used for file arguments, where you want to select only certain suffixes. For example, .vcf, .gvcf, etc.
• Suggestions: several default values to choose from

[patmagee]

Out of curiosity, isn't this achievable with the meta section which is now at both the task and the workflow level? everything else such as ui manipulations and what not, are up to the implementation and should not be a language feature. If you need the ability to nest more complex objects under the meta section, that I would say that is open for debate, but I think extending meta to allow for json like attributes might be a good thing. @orodeh @joshua-gould would that solve your problem

[orodeh]

@patmagee The parameter_meta section specifies per parameter attributes. More structure in that section would allow readable annotations for input/output parameters. Note that, currently, you can only annotate input parameters with a string.

The meta section is for global task/workflow annotations. Having more structure there would allow cleanly putting in per-task runtime policies, authorizations, and extra information. This example, JSON file has metadata for a DNAnexus applet, equivalent to a WDL task. Here some interesting pieces:

{
"name": "mapper",                 # Simple name; cannot be changed across versions of an app
"description": "This app...",     # ((optional) Longer description of what the app(let) does
 "developerNotes": "...",         # (optional) Detailed notes covering extra details
  "details": {                         # (optional) This can contain arbitrary metadata, but there are some
    "contactEmail": "alice@foo.edu"    #   key/value pairs that are recognized by web-sites 
  }                               
  "version": "0.0.1",                  # Apps only: version of the app that is being built
  "categories": ["Read Mapping"]
  "executionPolicy": {                 # (optional) Options governing job restart policy
      "restartOn": {
          "*": 3                       # Restart automatically up to 3 times for system errors
      }
    },
    "timeoutPolicy": {                 # (optional) User-specified timeout policy
        "hours": 4,
        "minutes": 30
      }
    }
}

[patmagee]

@orodeh so would extending the meta section to allow for json be all that you need?

[orodeh]

@patmagee JSON both in meta and parameter_meta.

Note that I tried modifying the hermes grammar a while ago, and found that embedding JSON in it wasn't so easy.

[geoffjentry]

This is exactly the sort of thing the meta and parameter_meta were created for. If they're somehow insufficient Id explore expanding them.

I'm very :-1: on adding these as general annotations (at least I'm consistent, I was vehement against a similar concept when it was being bandied about in the early CWL days)

[patmagee]

@orodeh @joshua-gould its seems as though the meta and parameter_meta are mostly sufficient. If there is something missing in the meta section I would add a separate PR for that. I am closing this issue since it seems to be solved

[orodeh]

@joshua-gould @patmagee, please open a new PR then, for enhancing the meta and parameter_meta sections.