search

Barski-lab / cwl-airflow

Python package to extend Airflow functionality with CWL1.1 support
https://barski-lab.github.io/cwl-airflow
Apache License 2.0
185 stars 32 forks source link

CWLJobDispatcher Fails #39

Closed manoskout closed 4 years ago

manoskout commented 4 years ago

The execution stops on CWLJobDispatcher When I trigger the workflow for execution, it stops in the first step. As we can see in the logs below, the error is KeyError.

To Reproduce Steps to reproduce the behavior:

  1. Create the workflow
  2. Trigger the workflow
  3. Click on the update button in Airflow UI
  4. The first step CWLJobDispatcher was failed

Logs

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 984, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/python3.7/site-packages/cwl_airflow/extensions/operators/cwljobdispatcher.py", line 49, in execute
    job=context["dag_run"].conf["job"],
KeyError: 'job'

Desktop :

manoskout commented 4 years ago

It was my mistake! I had a problem with my configuration JSON.

dagrooms52 commented 3 years ago

I'm seeing the same issue. Can you please share which configuration file had a problem? This is a problem with cwl-airflow's error messaging, it's unclear where to look for the issue.

dagrooms52 commented 3 years ago

This error doesn't come from CWL, Airflow, or the UI; the missing key comes from the DAG file. When triggering a job, pass a config JSON with the missing key as described in this tutorial.

In this case, passing {"job": "..."} should work.