Incompatible versions of cwltool/cwl-utils/schema-salad in dependencies

[pansapiens]

When installing janis-pipelines via pip, it looks like the current versions of cwltool and cwl-utils have conflicting pinned version ranges for schema-salad, such that an incompatible version is (sometimes?) installed.

(It seems this may be non-deterministic due to the way pip resolves dependencies :/ )

Reproducing, using the helloworld.py example:

conda create --yes -n janis python=3
conda activate janis
pip3 install janis-pipelines
janis translate helloworld.py wdl

While the pip install completes successfully, there is the error logged: ERROR: cwl-utils 0.5 has requirement schema-salad<7, but you'll have schema-salad 7.0.20200612160654 which is incompatible.

Running janus translate fails:

$ janis translate helloworld.py wdl
Traceback (most recent call last):
  ...snip...

    from janis_core.types import (
  File "/home/ubuntu/miniconda3/envs/janis2/lib/python3.8/site-packages/janis_core/types/__init__.py", line 1, in <module>
    from .data_types import PythonPrimitive, DataType
  File "/home/ubuntu/miniconda3/envs/janis2/lib/python3.8/site-packages/janis_core/types/data_types.py", line 17, in <module>
    import cwl_utils.parser_v1_0 as cwlgen
  File "/home/ubuntu/miniconda3/envs/janis2/lib/python3.8/site-packages/cwl_utils/parser_v1_0.py", line 29, in <module>
    from schema_salad.ref_resolver import Fetcher
ImportError: cannot import name 'Fetcher' from 'schema_salad.ref_resolver' (/home/ubuntu/miniconda3/envs/janis2/lib/python3.8/site-packages/schema_salad/ref_resolver.py)

If you downgrade schema-salad to a version cwl-utils is happy with, then it works:

$ pip3 install -U cwl-utils
...snip...

ERROR: cwltool 3.0.20200706173533 has requirement schema-salad<8,>=7, but you'll have schema-salad 6.0.20200601095207 which is incompatible.       
Installing collected packages: schema-salad
  Attempting uninstall: schema-salad
    Found existing installation: schema-salad 7.0.20200612160654
    Uninstalling schema-salad-7.0.20200612160654:
      Successfully uninstalled schema-salad-7.0.20200612160654
Successfully installed schema-salad-6.0.20200601095207

$ janis translate helloworld.py wdl
2020-07-09T16:52:46 [INFO]: Looking up digests for 1 containers (this might take a few minutes...)
..snip..
$ echo $?
0

I'm not sure if this then breaks the cwltool runner, I didn't test that.

Versions:

$ janis --version
--------------------  -------
janis-core            v0.9.19
janis-assistant       v0.9.18
janis-pipelines       v0.9.5
janis-unix            v0.9.2
janis-templates       v0.9.5
janis-bioinformatics  v0.9.14
--------------------  -------
$ conda --version
conda 4.8.2

$ python --version
Python 3.8.3

[illusional]

Hey @pansapiens, thanks for checking out the project and reporting the bug!

I've fixed CWL-Utils (https://github.com/common-workflow-language/cwl-utils/pull/26) and fixed this future janis installs (https://github.com/PMCC-BioinformaticsCore/janis-core/pull/37). This change should be released. (Your CWLTool might not work until schema salad is upgraded).

[pansapiens]

Thanks, working with janis-core v0.9.20.