search

astronomer / astro-provider-venv

Easily create and use Python Virtualenvs in Apache Airflow
Apache License 2.0
9 stars 4 forks source link

PYENV requirements can't be installed due to "the ssl module in Python is not available" #33

Open nugic opened 1 day ago

nugic commented 1 day ago

Astro CLI version = 1.29.0

Clean install: astro dev init

Dockerfile:

# syntax=quay.io/astronomer/airflow-extensions:v1

FROM quay.io/astronomer/astro-runtime:12.1.0

PYENV 3.12 boto boto-requirements.txt

boto-requirements.txt: boto3==1.35.21

Error message:

 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                     0.3s
 => => transferring dockerfile: 174B                                                                                                                                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                        0.3s
 => => transferring context: 113B                                                                                                                                                                                                                                        0.0s
 => resolve image config for quay.io/astronomer/airflow-extensions:v1                                                                                                                                                                                                    0.3s
 => docker-image://quay.io/astronomer/airflow-extensions:v1@sha256:382e851e253d442a40a5a89daed8e909658c64a833a65824099fa666940681fc                                                                                                                                      0.6s
 => => resolve quay.io/astronomer/airflow-extensions:v1@sha256:382e851e253d442a40a5a89daed8e909658c64a833a65824099fa666940681fc                                                                                                                                          0.0s
 => => sha256:382e851e253d442a40a5a89daed8e909658c64a833a65824099fa666940681fc 1.61kB / 1.61kB                                                                                                                                                                           0.0s
 => => sha256:c0b6661faa62c9b02938fa09140815d8f3d156bec54afa672ad11af9c257fd31 482B / 482B                                                                                                                                                                               0.0s
 => => sha256:9df383b147da87c3ca5eec40d2446409c2eafa190d6ffe21c28486f0e7073679 1.49kB / 1.49kB                                                                                                                                                                           0.0s
 => => sha256:a93ca41fd05e4c5aa111f833a712e241a29e24687f7653626f0b52568a08db2c 11.03MB / 11.03MB                                                                                                                                                                         0.3s
 => => extracting sha256:a93ca41fd05e4c5aa111f833a712e241a29e24687f7653626f0b52568a08db2c                                                                                                                                                                                0.2s
 => [astro] transform PYENV directives                                                                                                                                                                                                                                   0.2s
 => => transferring dockerfile: 174B                                                                                                                                                                                                                                     0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                        0.0s
 => [internal] load metadata for quay.io/astronomer/astro-runtime:12.1.0-base                                                                                                                                                                                            0.3s
 => [internal] load metadata for docker.io/library/python:3.12-slim-bullseye                                                                                                                                                                                             0.3s
 => [internal] load build context                                                                                                                                                                                                                                        0.5s
 => => transferring context: 15.92kB                                                                                                                                                                                                                                     0.0s
 => [stage-0  1/16] FROM quay.io/astronomer/astro-runtime:12.1.0-base@sha256:aafd80c649b94a0d1b62ca091446163798e1b4f17338886306981156f06b3cf1                                                                                                                            0.9s
 => => resolve quay.io/astronomer/astro-runtime:12.1.0-base@sha256:aafd80c649b94a0d1b62ca091446163798e1b4f17338886306981156f06b3cf1                                                                                                                                      0.0s
 => => sha256:aafd80c649b94a0d1b62ca091446163798e1b4f17338886306981156f06b3cf1 531B / 531B                                                                                                                                                                               0.0s
 => => sha256:2bef7c371f157502f77806adbb9f371e709a456e19764d44c45fe5cbbc3777eb 22.44kB / 22.44kB                                                                                                                                                                         0.0s
 => => sha256:e80eb48e07e4591ecd02264e53156d40b6d097f63ac4d652311c085da239a64d 5.61kB / 5.61kB                                                                                                                                                                           0.0s
 => [internal] settings cache mount permissions                                                                                                                                                                                                                          0.3s
 => FROM docker.io/library/python:3.12-slim-bullseye@sha256:34ba6fe9564c84d5f9252cbfff93f20e1ca325fa89d02cb0eb72ad73663c2314                                                                                                                                             2.9s
 => => resolve docker.io/library/python:3.12-slim-bullseye@sha256:34ba6fe9564c84d5f9252cbfff93f20e1ca325fa89d02cb0eb72ad73663c2314                                                                                                                                       0.0s
 => => sha256:5376e7a6ed0a47c1e8fe4309d97c4da3848b66a161194ba7158e62f05047fc57 1.75kB / 1.75kB                                                                                                                                                                           0.0s
 => => sha256:833db6e89e8b6f3b51dc469120bb8e5ce8f2c3751bfa625c89ce7edefd0abeb6 5.11kB / 5.11kB                                                                                                                                                                           0.0s
 => => sha256:6533c3eba3f3cd4c840877f9245b26929fc8c22a39f42c872aa314c32c6d654b 31.43MB / 31.43MB                                                                                                                                                                         0.3s
 => => sha256:6671afbf80d520b90e6bc20b00bf42d8c8b9b03f37e60a0bdb47ee01564df8b7 1.08MB / 1.08MB                                                                                                                                                                           0.1s
 => => sha256:035d07709dfb5588e1576091d0f994ee2c7ec39e89150803ff8ef975a9034596 12.63MB / 12.63MB                                                                                                                                                                         0.4s
 => => sha256:34ba6fe9564c84d5f9252cbfff93f20e1ca325fa89d02cb0eb72ad73663c2314 9.12kB / 9.12kB                                                                                                                                                                           0.0s
 => => sha256:102a6e7f7a21934a56851632b76315e0c2067b22256a4543c859a1c561bd957b 251B / 251B                                                                                                                                                                               0.1s
 => => extracting sha256:6533c3eba3f3cd4c840877f9245b26929fc8c22a39f42c872aa314c32c6d654b                                                                                                                                                                                0.8s
 => => extracting sha256:6671afbf80d520b90e6bc20b00bf42d8c8b9b03f37e60a0bdb47ee01564df8b7                                                                                                                                                                                0.2s
 => => extracting sha256:035d07709dfb5588e1576091d0f994ee2c7ec39e89150803ff8ef975a9034596                                                                                                                                                                                0.5s
 => => extracting sha256:102a6e7f7a21934a56851632b76315e0c2067b22256a4543c859a1c561bd957b                                                                                                                                                                                0.0s
 => [stage-0  2/16] COPY packages.txt .                                                                                                                                                                                                                                  0.5s
 => [stage-0  3/16] RUN if [[ -s packages.txt ]]; then     apt-get update && cat packages.txt | tr '\r\n' '\n' | xargs apt-get install -y --no-install-recommends     && apt-get clean     && rm -rf /var/lib/apt/lists/*;   fi                                          0.5s
 => [stage-0  4/16] COPY --link --from=python:3.12-slim-bullseye /usr/local/bin/*3.12* /usr/local/bin/                                                                                                                                                                   0.2s
 => [stage-0  5/16] COPY --link --from=python:3.12-slim-bullseye /usr/local/include/python3.12* /usr/local/include/python3.12                                                                                                                                            0.5s
 => [stage-0  6/16] COPY --link --from=python:3.12-slim-bullseye /usr/local/lib/pkgconfig/*3.12* /usr/local/lib/pkgconfig/                                                                                                                                               0.4s
 => [stage-0  7/16] COPY --link --from=python:3.12-slim-bullseye /usr/local/lib/*3.12*.so* /usr/local/lib/                                                                                                                                                               0.5s
 => [stage-0  8/16] COPY --link --from=python:3.12-slim-bullseye /usr/local/lib/python3.12 /usr/local/lib/python3.12                                                                                                                                                     0.8s
 => [stage-0  9/16] RUN /sbin/ldconfig /usr/local/lib                                                                                                                                                                                                                    0.5s
 => [stage-0 10/16] RUN mkdir -p /home/astro/.cache/pip /home/astro/.venv/boto                                                                                                                                                                                           0.5s 
 => [stage-0 11/16] COPY --chown=50000:0 boto-requirements.txt /home/astro/.venv/boto/requirements.txt                                                                                                                                                                   0.3s
 => [stage-0 12/16] RUN /usr/local/bin/python3.12 -m venv /home/astro/.venv/boto                                                                                                                                                                                         3.3s
 => ERROR [stage-0 13/16] RUN --mount=type=cache,uid=50000,gid=0,target=/home/astro/.cache/pip /home/astro/.venv/boto/bin/pip --cache-dir=/home/astro/.cache/pip install -r /home/astro/.venv/boto/requirements.txt                                                      9.0s
------                                                                                                                                                                                                                                                                        
 > [stage-0 13/16] RUN --mount=type=cache,uid=50000,gid=0,target=/home/astro/.cache/pip /home/astro/.venv/boto/bin/pip --cache-dir=/home/astro/.cache/pip install -r /home/astro/.venv/boto/requirements.txt:                                                                 
#25 0.543 + /home/astro/.venv/boto/bin/pip --cache-dir=/home/astro/.cache/pip install -r /home/astro/.venv/boto/requirements.txt                                                                                                                                              
#25 1.278 WARNING: Disabling truststore since ssl support is missing                                                                                                                                                                                                          
#25 1.314 WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
#25 1.329 Looking in indexes: https://pip.astronomer.io/v2/
#25 1.332 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /v2/boto3/
#25 1.833 WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /v2/boto3/
#25 2.833 WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /v2/boto3/
#25 4.834 WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /v2/boto3/
#25 8.835 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /v2/boto3/
#25 8.835 Could not fetch URL https://pip.astronomer.io/v2/boto3/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pip.astronomer.io', port=443): Max retries exceeded with url: /v2/boto3/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
#25 8.841 ERROR: Could not find a version that satisfies the requirement boto3==1.35.21 (from versions: none)
#25 8.848 WARNING: Disabling truststore since ssl support is missing
#25 8.851 WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
#25 8.853 Could not fetch URL https://pip.astronomer.io/v2/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pip.astronomer.io', port=443): Max retries exceeded with url: /v2/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
#25 8.857 ERROR: No matching distribution found for boto3==1.35.21
------
executor failed running [/bin/bash -o pipefail -e -u -x -c /home/astro/.venv/boto/bin/pip --cache-dir=/home/astro/.cache/pip install -r /home/astro/.venv/boto/requirements.txt]: exit code: 1
nugic commented 1 day ago

UPDATE: Using older version of airflow-extensions solved the problem: # syntax=quay.io/astronomer/airflow-extensions:v1.0.0