search

Barski-lab / cwl-airflow

Python package to extend Airflow functionality with CWL1.1 support
https://barski-lab.github.io/cwl-airflow
Apache License 2.0
185 stars 32 forks source link

Update readme #14

Closed michael-kotliar closed 6 years ago

michael-kotliar commented 6 years ago

FYI only

Mandatory

cwl-airflow package requires apache-airflow==1.9.0 that depends on 'psutil>=4.2.0, <5.0.0' (https://github.com/giampaolo/psutil) which in its turn requires python-dev (https://github.com/giampaolo/psutil/blob/master/INSTALL.rst#linux). So to avoid error psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or director it's necessary to install python-dev (python3-dev) to keep all header files and a static library for Python

Optional

If you are planning to use MySQL as backend you should consider installing mysqlclient>=1.3.6 (https://github.com/apache/incubator-airflow/blob/master/setup.py). In order to do this you might need to install (https://github.com/PyMySQL/mysqlclient-python) sudo apt-get install libmysqlclient-dev

Each job file run with the scheduler should include uid output_folder workflow Optionally tmp_folder If job file is run from cwl-airflow run these values could be omitted In this case they will be initilized with help of the arguments. Note, all present fields will not be overwritten --uid (default is randomly generated) --outdir (devault is current folder) Workflow is set as the first positional argument and doens't have any flags optionally --tmp