v1.1: Deprecate and make optional the declaration of many *Requirements

[mr-c]

• [ ] InlineJavascriptRequirement
• [ ] StepInputExpressionRequirement
• [ ] MultipleInputFeatureRequirement
• [ ] ScatterFeatureRequirement
• [ ] SubworkflowFeatureRequirement

Indicate that all are still optional features, but that users no longer have to declare them. Add conformance tests for using those features without declaring them.

[anton-khodak]

Add conformance tests for using those features without declaring them.

but there should be an ability to mark tests as optional first

[denis-yuen]

What would be the user experience be if they wrote/received a workflow that used an optional feature that was not supported by a particular platform? Do they find out by validation or runtime?

[inutano]

Is there any strong reason to do this? I know it's bothering to write lots of Requirement lines every time, but I think these should be explicitly declared. For people, like me, who is implementing a very tiny engine which aims to support CWL, these implicit requirement declarations make it difficult to handle errors of unsupported features.

[stain]

Agree that having explicit requirements is better for engines and validation.

It also makes it clearer for engines and engine users which features are implementable, rather than an engine seemingly not supporting particular deep configurations.

This seems to me like a quick attempt of syntactic sugar to avoid boilerplate. That should be visited in V2 by separating CWL syntax from model.

On 25 Jan 2018 8:58 am, "Tazro Inutano Ohta" notifications@github.com wrote:

Is there any strong reason to do this? I know it's bothering to write lots of Requirement lines every time, but I think these should be explicitly declared. For people, like me, who is implementing a very tiny engine which aims to support CWL, these implicit requirement declarations make it difficult to handle errors of unsupported features.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/common-workflow-language/common-workflow-language/issues/597#issuecomment-360402474, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPd5b611OeIechbDZsx8o27n_w5_QrJks5tOEI_gaJpZM4RgOBr .

[mr-c]

@denis-yuen It is already the case that these features are not required for declaring something to be compliant with CWL v1.0.x. We don't currently have any language about when to inform a user that their workflow uses an unsupported feature. Obviously the sooner the better.

The use of these features can be detected by simple parsing. We will include non-normative psuedo-code for use by implementers in the spec.

Here a snapshot as of 2018-01-25 of CWL implementation feature completeness with links to the test reports:

The following workflow systems/runners/platforms implement the entire spec: CWL reference runner (cwltool) toil-cwl-runner rabix executor

The following systems implement all mandatory features but lack complete conformance test coverage: airflow-cwl-runner Many scatter related tests fail arvados Four InitialWorkDirRequirement tests fail and the dockerOutputDirectory test also fails

The following systems have yet to pass the core CWL conformance tests on ci.commonwl.org: Galaxy Cromwell xenon-cwl-runner AWE ( not yet tested on ci.commonwl.org ) yacle ( not yet tested on ci.commonwl.org ) REANA (passing almost all tests, but not merged; not yet tested on ci.commonwl.org) Apache Taverna ( not yet tested on ci.commonwl.org ) Consonance ( not yet tested on ci.commonwl.org ) IBM's CWL on LSF implementation ( not yet released, not yet tested on ci.commonwl.org )

It was always our intent to drop the required declaration of some of these features once it became apparent which would be universally supported.

[denis-yuen]

Thanks for the in-depth explanation (I also learned that toil was supposed to be fully compliant!) Not sure if this is the right place for ideas, but I was not aware that requirements like InlineJavascriptRequirement or SubworkflowFeatureRequirement were optional. I suspect that it might be easy for others to also believe that these were universal.

If it is the case that some requirements are optional, there should be a standardized way of determining that a CWL workflow is, say Arvados+cwltool+toil compatible while a different workflow is compatible with Airflow+cwltool+toil .

I don't know if the best place is something like a --validate option on the standard cwl-runner interface or something like a central https://validator.w3.org/ But it might be frustrating to be in a future where one engine might tell you well into runtime

[tetron]

We definitely want to have a linter that reports portability problems. Somebody should work on that.

[inutano]

Hmm. A linter may help engine developers, but I suppose remaining these Requirements is the better way to remind users/developers a use of optional features. I think users also should be able to be aware of the use of features easily, even when they run a very long CWL workflow written by another user.

Those Requirement fields doesn't reduce readability of CWL, so it seems to me it just brings a small benefit to users, but a tricky problem to developers.

[tom-tan]

I guess this change will introduce few gains but more disadvantages.

First of all, the requirements is a good feature to indicate that the given CWL uses some optional features.

• Users can easily see whether the given CWL uses optional features or not.
• Developers can easily implement to reject unsupported features with good error messages.
  • All we have to do is to search for unsupported requirement in requirements field. It can be done with YAML or JSON parser!
  • For unsupported features, the error messages will be obvious: The requirement "XXXRequirement" is not supported by this workflow engine. Users can immediately know what's happen for the execution.

If requirements field for optional features is deprecated, it will introduce the following problems.

• Users side

  • It makes harder to see the given CWL uses optional features or not. For example, is it easy to see which optional features are used in this file without requirements?
  • Note: Linter tools may help this issue when he or she writes CWL. However such tools do not help when he or she reuses CWL written by other people. In that cases, we just pass it to the workflow engine without linting it.

• Developers side

  • It makes harder to check the given CWL uses unsupported features or not.
    • Now: Just checking requirements
    • After deprecation: We have to parse all contents in CWL file. Simple YAML and JSON parsers are not enough to do that. For example, we have to parse all $() to check whether it only contains parameter references or contains Javascript expressions.
  • (Related to users side): It makes harder to generate good error messages.
  • Now: The requirement "XXXRequirement" is not supported by this workflow engine or similar. Users will search "XXXRequirement" and will see what's wrong with it.
  • After deprecation: There are no unified keyword. The quality of the message will depends on workflow engine and it makes difficult for users to find out the problem. Error messages can be (e.g. for InlineJavascriptRequirement):
    • The requirement "InlineJavascriptRequirement" is not supported by this engine
    • Javascript expressions are not supported by this engine
    • Invalid expression: "..." (The engine will parse expressions as parameter references if it does not support InlineJavascriptExpression)
  • Although optional features are still optional, they will be considered as mandatory for users if they are implicit. As a result, all developers of workflow engine have to implement all optional features. It will prevent from developing new workflow engines.

As @stain said, this change will reduce some boilerplates when writing CWL. However, I am not sure there exist benefits than the disadvantages above.

[mr-c]

Thank you all for this great discussion. As a reminder, this is a proposed change for CWL v1.1, not CWL v1.0.x.

What if only the following two feature flags were made optional for v1.1?

• MultipleInputFeatureRequirement
• SubworkflowFeatureRequirement

[mr-c]

The point of making the declaration of these two feature optional is to be a signal that they will be required features to implement in a future release of the CWL standards.

[tom-tan]

It is good to make MultipleInputFeatureRequirement mandatory in the future release. It is common to use multiple inputs for a workflow and it does not introduce much complexity. It's worth to be mandatory.

I am not sure about SubworkflowFeatureRequirement. There exists many use cases for that but it will introduce an extra complexity to implement it. I guess it depends on the future direction of CWL, that is:

• CWL will support the major use cases as mandatory for users, or
• A mandatory part of CWL will be small to make easier to implement workflow engines and to remove entry barrier for developers.

[mr-c]

Deferring for more discussion