search

common-workflow-language / cwltool

Common Workflow Language reference implementation
https://cwltool.readthedocs.io/
Apache License 2.0
336 stars 231 forks source link

bug in cwltool when trying to run a tool with environment modules #603

Open hmenager opened 6 years ago

hmenager commented 6 years ago

Currently trying to use cwltool on our cluster using environment modules (@khillion and me), we have the following issue: the use of modules crashes cwltool, because it tries to store job_description dict and other informations in files opened in binary mode. here's the traceback that comes after trying it:

$ cwltool --beta-dependency-resolvers-configuration dependency-resolvers-conf.yml mafft.cwl mafft_solo.yml 
/pasteur/homes/kehillio/cwl/.venv/cwl/bin/cwltool 1.0.20171227212058
Resolved 'mafft.cwl' to 'file:///pasteur/homes/kehillio/cwl/cwl-phylogeny-test/mafft.cwl'
  Field `$schemas` contains undefined reference to `http://schema.org/docs/schema_org_rdfa.html`
mafft.cwl:3:1: unrecognized extension field `http://schema.org/about`.  Did you include a $schemas section?
mafft.cwl:3:1: unrecognized extension field `http://schema.org/keywords`.  Did you include a $schemas section?
mafft.cwl:3:1: unrecognized extension field `http://schema.org/name`.  Did you include a $schemas section?
mafft.cwl:3:1: unrecognized extension field `http://schema.org/publication`.  Did you include a $schemas section?
mafft.cwl:3:1: unrecognized extension field `http://schema.org/url`.  Did you include a $schemas section?
[job mafft.cwl] /local/scratch/tmpui_5lt4a$ mafft \
    --auto \
    /local/scratch/tmplluf2dg6/stg23f8872d-59d9-499b-921e-c89552bf397d/sequences.fa > /local/scratch/tmpui_5lt4a/mafft_alignment.fa
Exception while running job
Traceback (most recent call last):
  File "/pasteur/homes/kehillio/cwl/.venv/cwl/lib/python3.6/site-packages/cwltool/job.py", line 224, in _execute
    job_script_contents=job_script_contents,
  File "/pasteur/homes/kehillio/cwl/.venv/cwl/lib/python3.6/site-packages/cwltool/job.py", line 552, in _job_popen
    json.dump(job_description, f)
  File "/local/gensoft2/exe/Python/3.6.0/lib/python3.6/json/__init__.py", line 180, in dump
    fp.write(chunk)
TypeError: a bytes-like object is required, not 'str'
@hmenager

Your Environment

mr-c commented 6 years ago

Can we get a unit test for this? I'm guessing you have a CWL file with non-ASCII characters?