search

astronomer / astro-sdk

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
https://astro-sdk-python.rtfd.io/
Apache License 2.0
340 stars 42 forks source link

Unable to install packages with `astro-projects[snowflake,postgres]` #350

Closed tatiana closed 2 years ago

tatiana commented 2 years ago

Describe the bug Unable to create a virtual environment using requirements.txt:

astro-projects[postgres,snowflake]
beautifulsoup4==4.11.1
pandas
requests==2.27.1
twilio==7.0.0

Reported by: @jlaneve

Version

To Reproduce

  1. Create a virtualenv (e.g. using Python 3.9):

    python3.9 -m venv myenv

  2. Create a requirements.txt file with:

    astro-projects[postgres,snowflake]
beautifulsoup4==4.11.1
pandas
requests==2.27.1
twilio==7.0.0

  3. Install the requirements.txt file

    pip install -r requirements.txt

Expected behavior The user is able to install the dependencies.

Additional context

$ pip install -r requirements.txt 
Collecting astro-projects[postgres,snowflake]
  Downloading astro_projects-0.8.2-py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.2/54.2 KB 2.9 MB/s eta 0:00:00
Collecting beautifulsoup4==4.11.1
  Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.2/128.2 KB 7.2 MB/s eta 0:00:00
Collecting pandas
  Downloading pandas-1.4.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 2.4 MB/s eta 0:00:00
Collecting requests==2.27.1
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting twilio==7.0.0
  Downloading twilio-7.0.0.tar.gz (489 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 489.7/489.7 KB 1.1 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting soupsieve>1.2
  Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting pytz
  Using cached pytz-2022.1-py2.py3-none-any.whl (503 kB)
Collecting PyJWT<3.0.0,>=2.0.0
  Using cached PyJWT-2.3.0-py3-none-any.whl (16 kB)
Collecting python-frontmatter
  Using cached python_frontmatter-1.0.0-py3-none-any.whl (9.0 kB)
Collecting pyarrow
  Using cached pyarrow-7.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.7 MB)
Collecting SQLAlchemy<=1.3.24,>=1.3.18
  Using cached SQLAlchemy-1.3.24-cp39-cp39-manylinux2010_x86_64.whl (1.3 MB)
Collecting markupsafe<2.1.0,>=1.1.1
  Using cached MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB)
Collecting pandas
  Using cached pandas-1.3.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.5 MB)
Collecting smart-open
  Downloading smart_open-6.0.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.4/58.4 KB 208.4 kB/s eta 0:00:00
Collecting apache-airflow>=2.0
  Downloading apache_airflow-2.3.0-py3-none-any.whl (5.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 1.3 MB/s eta 0:00:00
Collecting apache-airflow-providers-postgres
  Using cached apache_airflow_providers_postgres-4.1.0-py3-none-any.whl (20 kB)
Collecting snowflake-sqlalchemy<=1.2.4,>=1.2.0
  Using cached snowflake_sqlalchemy-1.2.4-py2.py3-none-any.whl (29 kB)
Collecting apache-airflow-providers-snowflake
  Using cached apache_airflow_providers_snowflake-2.6.0-py3-none-any.whl (28 kB)
Collecting snowflake-connector-python[pandas]
  Downloading snowflake_connector_python-2.7.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.5/16.5 MB 2.5 MB/s eta 0:00:00
Collecting numpy>=1.17.3
  Using cached numpy-1.22.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting python-dateutil>=2.7.3
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting werkzeug>=1.0.1,~=1.0
  Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting graphviz>=0.12
  Downloading graphviz-0.20-py3-none-any.whl (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.0/47.0 KB 3.4 MB/s eta 0:00:00
Collecting deprecated>=1.2.13
  Using cached Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting jsonschema<5.0,>=3.2.0
  Downloading jsonschema-4.4.0-py3-none-any.whl (72 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.7/72.7 KB 3.3 MB/s eta 0:00:00
Collecting apache-airflow-providers-sqlite
  Using cached apache_airflow_providers_sqlite-2.1.3-py3-none-any.whl (16 kB)
Collecting jinja2<3.1,>=2.10.1
  Using cached Jinja2-3.0.3-py3-none-any.whl (133 kB)
Collecting croniter>=0.3.17
  Using cached croniter-1.3.4-py2.py3-none-any.whl (17 kB)
Collecting pendulum>=2.0
  Using cached pendulum-2.1.2-cp39-cp39-manylinux1_x86_64.whl (155 kB)
Collecting python-daemon>=2.2.4
  Using cached python_daemon-2.3.0-py2.py3-none-any.whl (35 kB)
Collecting httpx
  Using cached httpx-0.22.0-py3-none-any.whl (84 kB)
Collecting pygments>=2.0.1
  Downloading Pygments-2.12.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 3.1 MB/s eta 0:00:00
Collecting cryptography>=0.9.3
  Downloading cryptography-37.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.0/4.0 MB 3.2 MB/s eta 0:00:00
Collecting termcolor>=1.1.0
  Using cached termcolor-1.1.0-py3-none-any.whl
Collecting itsdangerous<2.0,>=1.1.0
  Using cached itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting lockfile>=0.12.2
  Using cached lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Collecting apache-airflow-providers-http
  Using cached apache_airflow_providers_http-2.1.2-py3-none-any.whl (21 kB)
Collecting dill>=0.2.2
  Using cached dill-0.3.4-py2.py3-none-any.whl (86 kB)
Collecting python-nvd3>=0.15.0
  Using cached python_nvd3-0.15.0-py3-none-any.whl
Collecting attrs<21.0,>=20.0
  Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting cattrs!=1.7.*,~=1.1
  Using cached cattrs-1.10.0-py3-none-any.whl (29 kB)
Collecting pluggy>=1.0
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting gunicorn>=20.1.0
  Using cached gunicorn-20.1.0-py3-none-any.whl (79 kB)
Collecting flask<2.0,>=1.1.0
  Using cached Flask-1.1.4-py2.py3-none-any.whl (94 kB)
Collecting connexion[flask,swagger-ui]>=2.10.0
  Downloading connexion-2.13.1-py2.py3-none-any.whl (95 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.1/95.1 KB 630.8 kB/s eta 0:00:00
Collecting blinker
  Using cached blinker-1.4-py3-none-any.whl
Collecting pathspec~=0.9.0
  Using cached pathspec-0.9.0-py2.py3-none-any.whl (31 kB)
Collecting flask-caching<2.0.0,>=1.5.0
  Using cached Flask_Caching-1.10.1-py3-none-any.whl (34 kB)
Collecting unicodecsv>=0.14.1
  Using cached unicodecsv-0.14.1-py3-none-any.whl
Collecting alembic<2.0,>=1.5.1
  Using cached alembic-1.7.7-py3-none-any.whl (210 kB)
Collecting typing-extensions>=3.7.4
  Using cached typing_extensions-4.2.0-py3-none-any.whl (24 kB)
Collecting cron-descriptor>=1.2.24
  Using cached cron_descriptor-1.2.24-py3-none-any.whl
Collecting flask-login<0.5,>=0.3
  Using cached Flask_Login-0.4.1-py2.py3-none-any.whl
Collecting psutil>=4.2.0
  Using cached psutil-5.9.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
Collecting python-slugify>=5.0
  Downloading python_slugify-6.1.2-py2.py3-none-any.whl (9.4 kB)
Collecting flask-session<=0.4.0,>=0.3.1
  Using cached Flask_Session-0.4.0-py2.py3-none-any.whl (7.5 kB)
Collecting rich>=12.3.0
  Downloading rich-12.3.0-py3-none-any.whl (232 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.1/232.1 KB 740.9 kB/s eta 0:00:00
Collecting colorlog<5.0,>=4.0.2
  Using cached colorlog-4.8.0-py2.py3-none-any.whl (10 kB)
Collecting flask-wtf<0.15,>=0.14.3
  Using cached Flask_WTF-0.14.3-py2.py3-none-any.whl (13 kB)
Collecting sqlalchemy-jsonfield>=1.0
  Using cached SQLAlchemy_JSONField-1.0.0-py3-none-any.whl (10 kB)
Collecting markdown>=3.0
  Using cached Markdown-3.3.6-py3-none-any.whl (97 kB)
Collecting apache-airflow-providers-ftp
  Using cached apache_airflow_providers_ftp-2.1.2-py3-none-any.whl (15 kB)
Collecting setproctitle>=1.1.8
  Downloading setproctitle-1.2.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (29 kB)
Collecting apache-airflow>=2.0
  Using cached apache_airflow-2.2.5-py3-none-any.whl (5.3 MB)
Collecting python3-openid~=3.2
  Using cached python3_openid-3.2.0-py3-none-any.whl (133 kB)
Collecting clickclick>=1.2
  Using cached clickclick-20.10.2-py2.py3-none-any.whl (7.4 kB)
Collecting wtforms<3.0.0
  Using cached WTForms-2.3.3-py2.py3-none-any.whl (169 kB)
Collecting jsonschema~=3.0
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting argcomplete<3.0,>=1.10
  Using cached argcomplete-2.0.0-py2.py3-none-any.whl (37 kB)
Collecting docutils<0.17
  Using cached docutils-0.16-py2.py3-none-any.whl (548 kB)
Collecting iso8601>=0.1.12
  Using cached iso8601-1.0.2-py3-none-any.whl (9.7 kB)
Collecting tabulate<0.9,>=0.7.5
  Using cached tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting colorlog<7.0,>=4.0.2
  Using cached colorlog-6.6.0-py2.py3-none-any.whl (11 kB)
Collecting python-slugify<5.0,>=3.0.0
  Using cached python_slugify-4.0.1-py2.py3-none-any.whl
Collecting lazy-object-proxy
  Using cached lazy_object_proxy-1.7.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61 kB)
Collecting marshmallow-oneofschema>=2.0.1
  Using cached marshmallow_oneofschema-3.0.1-py2.py3-none-any.whl (5.8 kB)
Collecting apache-airflow-providers-imap
  Using cached apache_airflow_providers_imap-2.2.3-py3-none-any.whl (16 kB)
Collecting flask-appbuilder==3.4.5
  Using cached Flask_AppBuilder-3.4.5-py3-none-any.whl (1.9 MB)
Collecting tenacity>=6.2.0
  Using cached tenacity-8.0.1-py3-none-any.whl (24 kB)
Collecting packaging>=14.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting email-validator<2,>=1.0.5
  Downloading email_validator-1.2.1-py2.py3-none-any.whl (22 kB)
Collecting colorama<1,>=0.3.9
  Using cached colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting marshmallow-enum<2,>=1.5.1
  Using cached marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB)
Collecting Flask-Babel<3,>=1
  Using cached Flask_Babel-2.0.0-py3-none-any.whl (9.3 kB)
Collecting Flask-JWT-Extended<4,>=3.18
  Using cached Flask_JWT_Extended-3.25.1-py2.py3-none-any.whl
Collecting prison<1.0.0,>=0.2.1
  Using cached prison-0.2.1-py2.py3-none-any.whl (5.8 kB)
Collecting click<9,>=6.7
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 KB 3.7 MB/s eta 0:00:00
Collecting marshmallow-sqlalchemy<0.27.0,>=0.22.0
  Using cached marshmallow_sqlalchemy-0.26.1-py2.py3-none-any.whl (15 kB)
INFO: pip is looking at multiple versions of apache-airflow to determine which version is compatible with other requirements. This could take a while.
Collecting apache-airflow>=2.0
  Using cached apache_airflow-2.2.4-py3-none-any.whl (5.3 MB)
Collecting flask-appbuilder==3.4.4
  Using cached Flask_AppBuilder-3.4.4-py3-none-any.whl (1.9 MB)
Collecting apache-airflow>=2.0
  Using cached apache_airflow-2.2.3-py3-none-any.whl (5.3 MB)
Collecting argcomplete~=1.10
  Using cached argcomplete-1.12.3-py2.py3-none-any.whl (38 kB)
Collecting pyyaml>=5.1
  Using cached PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)
Collecting cattrs<1.7.0,~=1.1
  Using cached cattrs-1.5.0-py3-none-any.whl (19 kB)
Collecting swagger-ui-bundle>=0.0.2
  Using cached swagger_ui_bundle-0.0.9-py3-none-any.whl (6.2 MB)
Collecting apache-airflow>=2.0
  Using cached apache_airflow-2.2.2-py3-none-any.whl (5.3 MB)
Collecting markupsafe<2.1.0,>=1.1.1
  Using cached MarkupSafe-1.1.1-cp39-cp39-manylinux2010_x86_64.whl (32 kB)
Collecting apache-airflow>=2.0
  Using cached apache_airflow-2.2.1-py3-none-any.whl (5.3 MB)
  Using cached apache_airflow-2.2.0-py3-none-any.whl (5.3 MB)
  Using cached apache_airflow-2.1.4-py3-none-any.whl (5.3 MB)
Collecting importlib-resources~=1.4
  Using cached importlib_resources-1.5.0-py2.py3-none-any.whl (21 kB)
Collecting apache-airflow>=2.0
  Using cached apache_airflow-2.1.3-py3-none-any.whl (5.3 MB)
  Using cached apache_airflow-2.1.2-py3-none-any.whl (5.2 MB)
  Using cached apache_airflow-2.1.1-py3-none-any.whl (5.2 MB)
  Using cached apache_airflow-2.1.0-py3-none-any.whl (5.3 MB)
  Using cached apache_airflow-2.0.2-py3-none-any.whl (4.6 MB)
Collecting cached-property~=1.5
  Using cached cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Collecting itsdangerous>=1.1.0
  Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting apache-airflow>=2.0
  Using cached apache_airflow-2.0.1-py3-none-any.whl (4.5 MB)
  Downloading apache_airflow-2.0.0-py3-none-any.whl (4.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 2.1 MB/s eta 0:00:00
Collecting colorlog==4.0.2
  Downloading colorlog-4.0.2-py2.py3-none-any.whl (17 kB)
INFO: pip is looking at multiple versions of pandas to determine which version is compatible with other requirements. This could take a while.
Collecting pandas
  Using cached pandas-1.3.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.5 MB)
INFO: pip is looking at multiple versions of astro-projects[postgres,snowflake] to determine which version is compatible with other requirements. This could take a while.
Collecting astro-projects[postgres,snowflake]
  Downloading astro_projects-0.8.1-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.0/54.0 KB 4.7 MB/s eta 0:00:00
  Downloading astro_projects-0.8.0-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.5/53.5 KB 193.0 kB/s eta 0:00:00
INFO: pip is looking at multiple versions of apache-airflow to determine which version is compatible with other requirements. This could take a while.
  Downloading astro_projects-0.7.0-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.2/53.2 KB 3.3 MB/s eta 0:00:00
  Downloading astro_projects-0.6.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.4/58.4 KB 3.6 MB/s eta 0:00:00
  Downloading astro_projects-0.5.1-py3-none-any.whl (55 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.0/55.0 KB 3.3 MB/s eta 0:00:00
WARNING: astro-projects 0.5.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.5.1 does not provide the extra 'snowflake'
Collecting apache-airflow-providers-google
  Downloading apache_airflow_providers_google-6.8.0-py3-none-any.whl (884 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 884.0/884.0 KB 2.1 MB/s eta 0:00:00
Collecting s3fs
  Using cached s3fs-2022.3.0-py3-none-any.whl (26 kB)
Collecting sqlalchemy-bigquery==1.3.0
  Using cached sqlalchemy_bigquery-1.3.0-py2.py3-none-any.whl (31 kB)
Collecting snowflake-sqlalchemy==1.2.0
  Using cached snowflake_sqlalchemy-1.2.0-py2.py3-none-any.whl (28 kB)
Collecting google-auth<3.0.0dev,>=1.25.0
  Downloading google_auth-2.6.6-py2.py3-none-any.whl (156 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 156.7/156.7 KB 4.2 MB/s eta 0:00:00
Collecting google-api-core>=1.30.0
  Downloading google_api_core-2.7.3-py3-none-any.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.6/114.6 KB 649.3 kB/s eta 0:00:00
Collecting google-cloud-bigquery<3.0.0dev,>=2.25.2
  Using cached google_cloud_bigquery-2.34.3-py2.py3-none-any.whl (206 kB)
Collecting future
  Using cached future-0.18.2-py3-none-any.whl
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Collecting markupsafe>=1.1.1
  Downloading MarkupSafe-2.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
INFO: pip is looking at multiple versions of sqlalchemy-bigquery to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of snowflake-sqlalchemy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of sqlalchemy to determine which version is compatible with other requirements. This could take a while.
Collecting astro-projects[postgres,snowflake]
  Downloading astro_projects-0.5.0-py3-none-any.whl (55 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.0/55.0 KB 808.8 kB/s eta 0:00:00
WARNING: astro-projects 0.5.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.5.0 does not provide the extra 'snowflake'
  Downloading astro_projects-0.4.0-py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.2/54.2 KB 622.4 kB/s eta 0:00:00
WARNING: astro-projects 0.4.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.4.0 does not provide the extra 'snowflake'
INFO: pip is looking at multiple versions of astro-projects[postgres,snowflake] to determine which version is compatible with other requirements. This could take a while.
  Using cached astro_projects-0.3.3-py3-none-any.whl (55 kB)
WARNING: astro-projects 0.3.3 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.3 does not provide the extra 'snowflake'
Collecting sqlalchemy<2.0.0
  Downloading SQLAlchemy-1.4.36-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 873.0 kB/s eta 0:00:00
  Downloading SQLAlchemy-1.4.9-cp39-cp39-manylinux2014_x86_64.whl (1.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 1.4 MB/s eta 0:00:00
Collecting pandas
  Downloading pandas-1.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 2.4 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of pyjwt to determine which version is compatible with other requirements. This could take a while.
Collecting PyJWT<3.0.0,>=2.0.0
  Downloading PyJWT-2.2.0-py3-none-any.whl (16 kB)
Collecting astro-projects[postgres,snowflake]
  Downloading astro_projects-0.3.2-py3-none-any.whl (55 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.1/55.1 KB 3.3 MB/s eta 0:00:00
WARNING: astro-projects 0.3.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.2 does not provide the extra 'snowflake'
  Using cached astro_projects-0.3.1-py3-none-any.whl (54 kB)
WARNING: astro-projects 0.3.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.1 does not provide the extra 'snowflake'
  Downloading astro_projects-0.2.2-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.6/53.6 KB 3.0 MB/s eta 0:00:00
WARNING: astro-projects 0.2.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.2 does not provide the extra 'snowflake'
  Downloading astro_projects-0.2.0-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.5/53.5 KB 2.7 MB/s eta 0:00:00
WARNING: astro-projects 0.2.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.0 does not provide the extra 'snowflake'
INFO: pip is looking at multiple versions of snowflake-sqlalchemy to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Using cached astro_projects-0.1.2-py3-none-any.whl (43 kB)
WARNING: astro-projects 0.1.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.2 does not provide the extra 'snowflake'
Collecting boto3==1.18.65
  Using cached boto3-1.18.65-py3-none-any.whl (131 kB)
Collecting s3transfer<0.6.0,>=0.5.0
  Using cached s3transfer-0.5.2-py3-none-any.whl (79 kB)
Collecting jmespath<1.0.0,>=0.7.1
  Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.22.0,>=1.21.65
  Using cached botocore-1.21.65-py3-none-any.whl (8.0 MB)
INFO: pip is looking at multiple versions of boto3 to determine which version is compatible with other requirements. This could take a while.
Collecting astro-projects[postgres,snowflake]
  Downloading astro_projects-0.1.1-py3-none-any.whl (43 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.8/43.8 KB 4.1 MB/s eta 0:00:00
WARNING: astro-projects 0.1.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.1 does not provide the extra 'snowflake'
INFO: pip is looking at multiple versions of pandas to determine which version is compatible with other requirements. This could take a while.
  Downloading astro_projects-0.1.0-py3-none-any.whl (43 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.7/43.7 KB 4.1 MB/s eta 0:00:00
WARNING: astro-projects 0.1.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.0 does not provide the extra 'snowflake'
  Downloading astro_projects-0.0.6-py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.5/42.5 KB 3.0 MB/s eta 0:00:00
WARNING: astro-projects 0.0.6 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.6 does not provide the extra 'snowflake'
  Downloading astro_projects-0.0.5-py3-none-any.whl (34 kB)
WARNING: astro-projects 0.0.5 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.5 does not provide the extra 'snowflake'
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Downloading astro_projects-0.0.4-py3-none-any.whl (34 kB)
WARNING: astro-projects 0.0.4 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.4 does not provide the extra 'snowflake'
WARNING: astro-projects 0.3.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.2 does not provide the extra 'snowflake'
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
WARNING: astro-projects 0.3.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.1 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.1 does not provide the extra 'snowflake'
INFO: pip is looking at multiple versions of boto3 to determine which version is compatible with other requirements. This could take a while.
WARNING: astro-projects 0.1.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.6 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.6 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.5 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.5 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.4 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.4 does not provide the extra 'snowflake'
Collecting PyJWT<3.0.0,>=2.0.0
  Downloading PyJWT-2.1.0-py3-none-any.whl (16 kB)
WARNING: astro-projects 0.3.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.3.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.1 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.2 does not provide the extra 'snowflake'
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
WARNING: astro-projects 0.1.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.1 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.6 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.6 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.5 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.5 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.4 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.4 does not provide the extra 'snowflake'
  Downloading PyJWT-2.0.1-py3-none-any.whl (15 kB)
WARNING: astro-projects 0.3.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.3.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.1 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.1 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.6 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.6 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.5 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.5 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.4 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.4 does not provide the extra 'snowflake'
  Downloading PyJWT-2.0.0-py3-none-any.whl (15 kB)
WARNING: astro-projects 0.3.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.3.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.3.1 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.2.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.2.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.2 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.2 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.1 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.1 does not provide the extra 'snowflake'
WARNING: astro-projects 0.1.0 does not provide the extra 'postgres'
WARNING: astro-projects 0.1.0 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.6 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.6 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.5 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.5 does not provide the extra 'snowflake'
WARNING: astro-projects 0.0.4 does not provide the extra 'postgres'
WARNING: astro-projects 0.0.4 does not provide the extra 'snowflake'
INFO: pip is looking at multiple versions of twilio to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of beautifulsoup4 to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 5), astro-projects and requests==2.27.1 because these package versions have conflicting dependencies.

The conflict is caused by:
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.2.3 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.2.2 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.2.1 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.2.0 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.1.4 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.1.3 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.1.2 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.1.1 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.1.0 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.0.2 depends on pyjwt<2
    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.0.1 depends on pyjwt<2
    The user requested requests==2.27.1
    twilio 7.0.0 depends on requests>=2.0.0
    apache-airflow 2.0.0 depends on requests<2.24.0 and >=2.20.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
tatiana commented 2 years ago

@jlaneve I was able to reproduce the issue.

It seems that if you used a different version of Twilio it would work (twilio-6.63.2), would this be a workaround for your problem?

The problem you are facing seems to be related to Airflow 2.2.3 dependencies and Twilio 7.0.0:

    twilio 7.0.0 depends on PyJWT<3.0.0 and >=2.0.0
    apache-airflow 2.2.3 depends on pyjwt<2

I noticed that due to the SQLAlchemy upgrade in Airflow 2.3.0, the Python Astro SDK is not compatible with the latest version of Airflow. I'm logging a separate issue for this.

kaxil commented 2 years ago

You will need the following requirements, this is because of conflicting dependency between Flask-Appbuilder 3.4.5 (https://github.com/dpgaspar/Flask-AppBuilder/blob/v3.4.5/setup.py#L65) and Twilio 7.0.0, which is downgrading Twilio to 6.63.2 is needed. Once Airflow bumps Fab to >=4 it will be fine as it bumped PyJWT then: https://github.com/dpgaspar/Flask-AppBuilder/releases/tag/v4.0.0

astro-sdk-python[postgres,snowflake]
beautifulsoup4>=4.11.1
pandas
requests>=2.27.1
twilio>=6.63.2