idealista / airflow-role

Ansible role to install Apache Airflow
Apache License 2.0
83 stars 52 forks source link

[FEATURE] add async / poll params on Airflow | Initializing DB to fix hanging task #28

Closed ataki closed 6 years ago

ataki commented 6 years ago

First, thanks for making this project open source, it's saved me a bunch of time in getting airflow successfully deployed.

I thought I'd note this down since others may run into this issue.

My environment:

---
- hosts: someserver
  roles:
    - idealista/airflow-role

I had an uninstalled import in one of my dags, which I expected to cause ansible to fail on the task "Airflow | Initializing DB", since it has to import the dags as part of this process. Instead, the exit signal is not received by ansible and the task hangs.

I discovered this by adding this to my task definition:

async: 60
poll: 60

I'm fairly new to ansible, and not suggesting that async is the best way to handle this, but the expected behavior on a bad import is that the ansible task should fail instead of hang.

jnogol commented 6 years ago

Hi @ataki!

Thank you for your nice words, we're happy to know that our role is helpful.

I've modified the task in a new branch, using command and environment and tried it and it seems to work. Please checkout to bugs/28 and tell us if it works for you too.

Regards, Jose

ataki commented 6 years ago

Hey @jnogol,

Great! Can verify that running on the same setup I had before now works as expected - the task doesn't hang and the db is initialized.

jnogol commented 6 years ago

Cool, we're gonna merge this to develop and include it in our next release. Thanks for your feedback!

jnogol commented 6 years ago

Hey @ataki,

We've just released v1.3.2 that fixes this problem.

Regards, Jose