wtsi-hgi / gatk-cwl-generator

Generates CWL files from the GATK documentation
MIT License
7 stars 1 forks source link

Add annotation enum #14

Closed ThomasHickman closed 6 years ago

ThomasHickman commented 6 years ago

The annotations parameter can only accept certain parameters. This tool should generate an enum with a symbols object containing those options.

sersorrel commented 6 years ago

I have an implementation of this, but common-workflow-language/cwltool#576 suggests that cwltool currently can't handle arrays of enum (meaning that merging it would make the CWL files unusable).

mr-c commented 6 years ago

Hello @anowlcalledjosh , that issue is just about providing the value via cwltool's auto-generated command line interface. You can still use arrays of enums via the JSON/YAML input object or whatever methods other CWL executors use (graphical, for example).

sersorrel commented 6 years ago

It looks like arrays of enums don't work even with the normal YAML input @mr-c:

inp.cwl:

cwlVersion: v1.0
class: CommandLineTool
baseCommand: echo
inputs:
  args:
    type:
      - type: array
        items:
          type: enum
          symbols:
            - foo
            - bar
    inputBinding:
      position: 1
outputs: []

job.yml:

args: [foo]

cwltool inp.cwl job.yml:

/Users/jh36/venv/bin/cwltool 1.0.20180711112827
Resolved 'inp.cwl' to 'file:///Users/jh36/inp.cwl'
Got workflow error
Traceback (most recent call last):
  File "/Users/jh36/venv/lib/python3.6/site-packages/avro/schema.py", line 493, in __init__
    raise SchemaParseException(fail_msg)
  File "/Users/jh36/venv/lib/python3.6/site-packages/avro/schema.py", line 740, in make_avsc_object
    doc = json_data.get('doc')
  File "/Users/jh36/venv/lib/python3.6/site-packages/avro/schema.py", line 457, in __init__
    self.set_prop('symbols', symbols)
  File "/Users/jh36/venv/lib/python3.6/site-packages/avro/schema.py", line 298, in __init__
    self.set_prop('name', name)
  File "/Users/jh36/venv/lib/python3.6/site-packages/avro/schema.py", line 275, in add_name
    return to_add
avro.schema.SchemaParseException: The name "argsfdef0918-6e9a-44fd-81f1-a94061a729a1" is already in use.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jh36/venv/lib/python3.6/site-packages/cwltool/executors.py", line 141, in run_jobs
    for job in jobiter:
  File "/Users/jh36/venv/lib/python3.6/site-packages/cwltool/command_line_tool.py", line 373, in job
    builder = self._init_job(job_order, runtimeContext)
  File "/Users/jh36/venv/lib/python3.6/site-packages/cwltool/process.py", line 654, in _init_job
    discover_secondaryFiles=getdefault(runtimeContext.toplevel, False)))
  File "/Users/jh36/venv/lib/python3.6/site-packages/cwltool/builder.py", line 260, in bind_input
    bindings.extend(self.bind_input(f, datum[f["name"]], lead_pos=lead_pos, tail_pos=f["name"], discover_secondaryFiles=discover_secondaryFiles))
  File "/Users/jh36/venv/lib/python3.6/site-packages/cwltool/builder.py", line 231, in bind_input
    avsc = AvroSchemaFromJSONData(t, self.names)
  File "/Users/jh36/venv/lib/python3.6/site-packages/avro/schema.py", line 750, in make_avsc_object
    return MapSchema(values, names, other_props)
  File "/Users/jh36/venv/lib/python3.6/site-packages/avro/schema.py", line 496, in __init__
avro.schema.SchemaParseException: Items schema (CommentedMap([('type', 'enum'), ('symbols', ['foo', 'bar']), ('name', 'argsfdef0918-6e9a-44fd-81f1-a94061a729a1')])) not a valid Avro schema: The name "argsfdef0918-6e9a-44fd-81f1-a94061a729a1" is already in use. (known names: ['File', 'File_class', 'Directory', 'Directory_class', 'Any', 'input_record_schema', 'argsfdef0918-6e9a-44fd-81f1-a94061a729a1', 'outputs_record_schema'])
Workflow error, try again with --debug for more information:
Items schema (CommentedMap([('type', 'enum'), ('symbols', ['foo', 'bar']), ('name', 'argsfdef0918-6e9a-44fd-81f1-a94061a729a1')])) not a valid Avro schema: The name "argsfdef0918-6e9a-44fd-81f1-a94061a729a1" is already in use. (known names: ['File', 'File_class', 'Directory', 'Directory_class', 'Any', 'input_record_schema', 'argsfdef0918-6e9a-44fd-81f1-a94061a729a1', 'outputs_record_schema'])
mr-c commented 6 years ago

I can confirm this as a bug in cwltool, thanks. @anowlcalledjosh can you file an issue?

Here's a work around:

cwlVersion: v1.0
class: CommandLineTool
baseCommand: echo
requirements:
  SchemaDefRequirement:
    types:
      - type: enum
        name: my_enum
        symbols: [ foo, bar ]

inputs:
  args:
    type: my_enum[]
    inputBinding:
      position: 1
outputs: []
sersorrel commented 6 years ago

Filed as common-workflow-language/cwltool#821.

That workaround looks like it will make my code a lot nicer anyway, thanks!