Update to Airflow 1.10.0 and latest cwltool

Barski-lab / cwl-airflow

Python package to extend Airflow functionality with CWL1.1 support

https://barski-lab.github.io/cwl-airflow

Apache License 2.0

185 stars 32 forks source link

Update to Airflow 1.10.0 and latest cwltool #18

Closed michael-kotliar closed 5 years ago

michael-kotliar commented 6 years ago

Currently fails most of the conformance tests with Airflow 1.10.0

canlar commented 5 years ago

Hi, if click create user in airflwo i get error. Solution: update Flask-Admin

Test airflow 1.10: i upgrade to airflow 1.10 airflow upgradedb is the one needed step to execute cwl in airflwo 1.10.

Do i need other steps ? How i can use AWS Batch ?

Best regards canlar

canlar commented 5 years ago

I edited airflow.cfg celery Parameter. celeryd_concurrency >>> worker_concurrency ...

michael-kotliar commented 5 years ago

Hi @canlar, Thank you for contributing to the development. Could you please provide more information about the system you are planning to build on AWS Batch? Meanwhile, did you consider using Google Cloud Composer? It looks like they have already done part of the work towards running Airflow in cloud.