search

puckel / docker-airflow

Docker Apache Airflow
Apache License 2.0
3.77k stars 532 forks source link

Dockerfile not consistent with latest image? #353

Open hxu-lift opened 5 years ago

hxu-lift commented 5 years ago

When I use image from docker hub: puckel/docker-airflow:1.10.2 and docker-compose-LocalExecutor.yml for docker-compose up I got no issue running the airflow web server.

However when I try to build image from the Dockerfile and run the same docker-compose file, I got this:

webserver_1  | Traceback (most recent call last):
webserver_1  |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1  |     from airflow import configuration
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/airflow/__init__.py", line 36, in <module>
webserver_1  |     from airflow import settings, configuration as conf
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 266, in <module>
webserver_1  |     configure_orm()
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 188, in configure_orm
webserver_1  |     engine = create_engine(SQL_ALCHEMY_CONN, **engine_args)
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/__init__.py", line 431, in create_engine
webserver_1  |     return strategy.create(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
webserver_1  |     dbapi = dialect_cls.dbapi(**dbapi_args)
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 599, in dbapi
webserver_1  |     import psycopg2
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py", line 50, in <module>
webserver_1  |     from psycopg2._psycopg import (                     # noqa
webserver_1  | ImportError: libpq.so.5: cannot open shared object file: No such file or directory
webserver_1  | Traceback (most recent call last):
webserver_1  |   File "/usr/local/bin/airflow", line 21, in <module>
webserver_1  |     from airflow import configuration
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/airflow/__init__.py", line 36, in <module>
webserver_1  |     from airflow import settings, configuration as conf
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 266, in <module>
webserver_1  |     configure_orm()
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 188, in configure_orm
webserver_1  |     engine = create_engine(SQL_ALCHEMY_CONN, **engine_args)
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/__init__.py", line 431, in create_engine
webserver_1  |     return strategy.create(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
webserver_1  |     dbapi = dialect_cls.dbapi(**dbapi_args)
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 599, in dbapi
webserver_1  |     import psycopg2
webserver_1  |   File "/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py", line 50, in <module>
webserver_1  |     from psycopg2._psycopg import (                     # noqa
webserver_1  | ImportError: libpq.so.5: cannot open shared object file: No such file or directory

Is there some configurations I am missing?

siddardha7 commented 5 years ago

See #342

Edit the Dockerfile to change the airflow version to 1.10.1 or add that dependency.

shreyapatni commented 5 years ago

Install library libpq5 after buildDeps as it is getting removed later:

&& apt-get purge --auto-remove -yqq $buildDeps \
bryan3189 commented 5 years ago

@shreyapatni Can you provide an example of where you are installing libpq5? Thanks!

thierryturpin commented 5 years ago

It's because of the psycopg2 release... Created #370 for it.