pepkit / looper

A job submitter for Portable Encapsulated Projects
http://looper.databio.org
BSD 2-Clause "Simplified" License
20 stars 7 forks source link

Example pipeline interface does not validate #287

Closed afrendeiro closed 4 years ago

afrendeiro commented 4 years ago

I tried validating the pipeline interface provided in the example without success: http://looper.databio.org/en/latest/pipeline-interface-specification/#Overview%20of%20pipeline%20interface%20components

$ cat example_interface.yaml
pipeline_name: "RRBS"
pipeline_type: "sample"
# path: "path/to/rrbs.py"
# input_schema: "path/to/rrbs_schema.yaml"
command_template: "{pipeline.path} --input {sample.data_path}"
$ eido validate example_interface.yaml -s http://schema.databio.org/pipelines/pipeline_interface.yaml -e -c
Traceback (most recent call last):
  File "//home/afr/.local/bin/eido", line 8, in <module>
    sys.exit(main())
  File "/home/afr/.local/lib/python3.8/site-packages/eido/eido.py", line 323, in main
    validate_config(p, args.schema, args.exclude_case)
  File "/home/afr/.local/lib/python3.8/site-packages/eido/eido.py", line 221, in validate_config
    _validate_object(project_dict, schema_cpy, exclude_case)
  File "/home/afr/.local/lib/python3.8/site-packages/eido/eido.py", line 159, in _validate_object
    raise jsonschema.exceptions.ValidationError(e.message)
jsonschema.exceptions.ValidationError: 'pipeline_name' is a required property

Similar output for other pipeline interfaces that look valid on manual inspection.

stolarczyk commented 4 years ago

eido is a validation tool for PEPs, not pipeline interfaces. However, the pipeline interface is indeed validated against a schema every time it's read, but within looper: https://github.com/pepkit/looper/blob/ba5e3231c9012ec5459b047c663d0c0db0dedbf1/looper/pipeline_interface.py#L51

afrendeiro commented 4 years ago

Ah, alright my bad. Thanks!

nsheff commented 4 years ago

the pipeline interfaces will validate using straight-up JSON-schema... eido isn't necessary for that.

but this does bring up a point -- there's not an easy way that I could find to validate the yaml file (the jsonschema command expects it to be in json format). So maybe the eido command could have a simple function that would just wrap jsonschema. we're doing that already...