common-workflow-language / cwl-v1.3

Apache License 2.0
4 stars 4 forks source link

Supporting recursive expansion of `typeDSL` explicitly #10

Open tom-tan opened 1 year ago

tom-tan commented 1 year ago

Currently cwltool does not support the following CWL document.

nestedTypeDSL.cwl

cwlVersion: v1.2
class: CommandLineTool
baseCommand: echo
inputs:
    inp: File[][]
outputs: []

We expect it is valid but it is not:

$ cwltool --validate nestedTypeDSL.cwl 
INFO /home/vscode/.local/bin/cwltool 3.1.20230601100705
INFO Resolved 'nestedTypeDSL.cwl' to 'file:///workspaces/cwl-v1.2/nestedTypeDSL.cwl'
ERROR Tool definition failed validation:
nestedTypeDSL.cwl:4:1: checking field 'inputs'
nestedTypeDSL.cwl:5:5:   checking object 'nestedTypeDSL.cwl#inp'
                           Field 'type' references unknown identifier 'File[][]', tried
                           file:///workspaces/cwl-v1.2/nestedTypeDSL.cwl#File[][]

It happens because cwltool (more precisely, schema-salad codegen) expand File[][] into { type: array, items: File[] } but its element type is not expanded.

Obviously users expect that File[][] is supported by the spec. On the other hand, the spec of SALAD says:

If the type ends with square brackets [] it is expanded to an array with items of the preceeding type symbol

That is, the spec does not require to expand the array types recursively and thus the current behavior of cwltool is valid in CWL v1.2.

It would be nice if SALAD explicitly requires to expand typeDSL recursively.

Note: To support this feature, we have to fix schema-salad codegen.

tetron commented 1 year ago

Related: https://github.com/common-workflow-language/cwl-v1.2/issues/254

The intention is that, going from right to left, each symbol [] or ? logically wraps the type specification to the left of it.