search

developmentseed / eoapi-cdk

AWS CDK constructs for deploying eoAPI
https://developmentseed.org/eoapi-cdk
11 stars 4 forks source link

Docker build failure of titiler-pgstac-api #107

Closed bransonf closed 2 months ago

bransonf commented 2 months ago

Using the TitilerPgstacApiLambda construct via the Python package. 

titiler_lambda = TitilerPgstacApiLambda(
            self,
            "titiler-pgstac-api-ecr",
            api_env=titiler_env,
            vpc=vpc,
            db=pgstac_db.db,
            db_secret=pgstac_db.pgstac_secret,
            subnet_selection=aws_ec2.SubnetSelection(
                subnet_type=(
                    aws_ec2.SubnetType.PUBLIC
                    if app_config.public_db_subnet
                    else aws_ec2.SubnetType.PRIVATE_WITH_EGRESS
                )
            ),
            lambda_function_options={
                "code": aws_lambda.Code.from_ecr_image(
                    repository=titiler_ecr_repository,
                    tag=app_config.server_docker_image_tag,
                ),
                "allow_public_subnet": True,
                "handler": aws_lambda.Handler.FROM_IMAGE,
                "runtime": aws_lambda.Runtime.FROM_IMAGE,
                "memory_size": 3008,
                "timeout": Duration.seconds(30),
                "log_retention": aws_logs.RetentionDays.ONE_WEEK,
            },
            ...
)

AWS CDK attempts to build the Default image (https://github.com/developmentseed/eoapi-cdk/blob/7536c551d0f2098f2ef205cd698c740e24cd294d/lib/titiler-pgstac-api/runtime/Dockerfile)

and fails during compilation of numexpr with:

g++ -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DNPY_TARGET_VERSION=NPY_1_23_API_VERSION -I/tmp/pip-build-env-0fjxgqvb/overlay/lib/python3.11/site-packages/numpy/_core/include -I/var/lang/include/python3.11 -c numexpr/interpreter.cpp -o build/temp.linux-x86_64-cpython-311/numexpr/interpreter.o
error: command 'g++' failed: No such file or directory

Notably, other images (e.g. https://github.com/developmentseed/eoapi-cdk/blob/7536c551d0f2098f2ef205cd698c740e24cd294d/lib/tipg-api/runtime/Dockerfile) use:

RUN python -m pip install -r requirements.txt "mangum>=0.14,<0.15" -t /asset --no-binary pydantic

whereas titiler-pgstac uses:

RUN python -m pip install -r requirements.txt "mangum>=0.14,<0.15" -t /asset

I reproduced the build failure on versions 6.0.2, 7.0.0 and 7.2.0

Could probably also be solved via https://github.com/developmentseed/eoapi-cdk/issues/87, given that the CDK is attempting to build the default image even though it is overridden.

Thanks for any help you might be able to offer here!

hrodmn commented 2 months ago

Thanks for the bug report @bransonf! Here is the full error log that I see when I try to build the docker image locally:

henry@quercus:~/workspace/devseed/eoapi-cdk/lib/titiler-pgstac-api$ docker build . --build-arg PYTHON_VERSION=3.11 -f runtime/Dockerfile
[+] Building 15.9s (9/15)                                                             docker:default
 => [internal] load build definition from Dockerfile                                            0.0s
 => => transferring dockerfile: 854B                                                            0.0s
 => WARN: InvalidDefaultArgInFrom: Default value for ARG public.ecr.aws/lambda/python:${PYTHON  0.0s
 => WARN: FromPlatformFlagConstDisallowed: FROM --platform flag should not use constant value   0.0s
 => [internal] load metadata for public.ecr.aws/lambda/python:3.11                              0.7s
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 2B                                                                 0.0s
 => [ 1/11] FROM public.ecr.aws/lambda/python:3.11@sha256:1816be4e08ecf134370c69d05ad3c9c72c75  0.0s
 => [internal] load build context                                                               0.0s
 => => transferring context: 2.08kB                                                             0.0s
 => CACHED [ 2/11] WORKDIR /tmp                                                                 0.0s
 => CACHED [ 3/11] RUN python -m pip install pip -U                                             0.0s
 => [ 4/11] COPY runtime/requirements.txt requirements.txt                                      0.0s
 => ERROR [ 5/11] RUN python -m pip install -r requirements.txt "mangum>=0.14,<0.15" -t /asse  15.1s
------
 > [ 5/11] RUN python -m pip install -r requirements.txt "mangum>=0.14,<0.15" -t /asset:
0.688 Collecting mangum<0.15,>=0.14
0.786   Downloading mangum-0.14.1-py3-none-any.whl.metadata (3.1 kB)
0.836 Collecting titiler.pgstac==1.2.2 (from -r requirements.txt (line 1))
0.886   Downloading titiler.pgstac-1.2.2-py3-none-any.whl.metadata (6.4 kB)
0.934 Collecting psycopg[binary,pool] (from -r requirements.txt (line 2))
0.953   Downloading psycopg-3.2.1-py3-none-any.whl.metadata (4.2 kB)
1.000 Collecting geojson-pydantic~=1.0 (from titiler.pgstac==1.2.2->-r requirements.txt (line 1))
1.025   Downloading geojson_pydantic-1.1.0-py3-none-any.whl.metadata (4.1 kB)
1.065 Collecting pydantic-settings~=2.0 (from titiler.pgstac==1.2.2->-r requirements.txt (line 1))
1.090   Downloading pydantic_settings-2.4.0-py3-none-any.whl.metadata (3.5 kB)
1.298 Collecting pydantic<3.0,>=2.4 (from titiler.pgstac==1.2.2->-r requirements.txt (line 1))
1.327   Downloading pydantic-2.8.2-py3-none-any.whl.metadata (125 kB)
1.431 Collecting titiler.core<0.18,>=0.17.0 (from titiler.pgstac==1.2.2->-r requirements.txt (line 1))
1.475   Downloading titiler.core-0.17.3-py3-none-any.whl.metadata (4.0 kB)
1.528 Collecting titiler.mosaic<0.18,>=0.17.0 (from titiler.pgstac==1.2.2->-r requirements.txt (line 1))
1.576   Downloading titiler.mosaic-0.17.3-py3-none-any.whl.metadata (2.6 kB)
1.619 Collecting typing-extensions (from mangum<0.15,>=0.14)
1.646   Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
1.825 Collecting psycopg-binary==3.2.1 (from psycopg[binary,pool]->-r requirements.txt (line 2))
1.845   Downloading psycopg_binary-3.2.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.8 kB)
1.884 Collecting psycopg-pool (from psycopg[binary,pool]->-r requirements.txt (line 2))
1.903   Downloading psycopg_pool-3.2.2-py3-none-any.whl.metadata (2.6 kB)
1.952 Collecting annotated-types>=0.4.0 (from pydantic<3.0,>=2.4->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
1.972   Downloading annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
2.560 Collecting pydantic-core==2.20.1 (from pydantic<3.0,>=2.4->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
2.579   Downloading pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
2.634 Collecting python-dotenv>=0.21.0 (from pydantic-settings~=2.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
2.653   Downloading python_dotenv-1.0.1-py3-none-any.whl.metadata (23 kB)
2.716 Collecting fastapi>=0.107.0 (from titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
2.734   Downloading fastapi-0.112.1-py3-none-any.whl.metadata (27 kB)
2.773 Collecting jinja2<4.0.0,>=2.11.2 (from titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
2.799   Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
2.835 Collecting morecantile<6.0,>=5.0 (from titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
2.857   Downloading morecantile-5.3.1-py3-none-any.whl.metadata (6.9 kB)
2.994 Collecting numpy (from titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.013   Downloading numpy-2.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
3.160 Collecting rasterio (from titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.178   Downloading rasterio-1.3.10-cp311-cp311-manylinux2014_x86_64.whl.metadata (14 kB)
3.232 Collecting rio-tiler<7.0,>=6.3.0 (from titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.250   Downloading rio_tiler-6.6.1-py3-none-any.whl.metadata (11 kB)
3.338 Collecting simplejson (from titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.358   Downloading simplejson-3.19.3-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
3.390 Collecting cogeo-mosaic<8.0,>=7.0 (from titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.410   Downloading cogeo_mosaic-7.1.0-py3-none-any.whl.metadata (17 kB)
3.450 Collecting attrs (from cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.469   Downloading attrs-24.2.0-py3-none-any.whl.metadata (11 kB)
3.507 Collecting cachetools (from cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.527   Downloading cachetools-5.5.0-py3-none-any.whl.metadata (5.3 kB)
3.566 Collecting httpx (from cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.586   Downloading httpx-0.27.0-py3-none-any.whl.metadata (7.2 kB)
3.675 Collecting shapely<3.0,>=2.0 (from cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.694   Downloading shapely-2.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.0 kB)
3.724 Collecting supermorecado (from cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.745   Downloading supermorecado-0.1.2-py3-none-any.whl.metadata (8.2 kB)
3.787 Collecting starlette<0.39.0,>=0.37.2 (from fastapi>=0.107.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.808   Downloading starlette-0.38.2-py3-none-any.whl.metadata (5.9 kB)
3.873 Collecting MarkupSafe>=2.0 (from jinja2<4.0.0,>=2.11.2->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.893   Downloading MarkupSafe-2.1.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
3.954 Collecting pyproj~=3.1 (from morecantile<6.0,>=5.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
3.976   Downloading pyproj-3.6.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
4.017 Collecting color-operations (from rio-tiler<7.0,>=6.3.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
4.036   Downloading color_operations-0.1.5-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.7 kB)
4.093 Collecting numexpr (from rio-tiler<7.0,>=6.3.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
4.113   Downloading numexpr-2.10.1.tar.gz (101 kB)
4.138   Installing build dependencies: started
8.476   Installing build dependencies: finished with status 'done'
8.477   Getting requirements to build wheel: started
8.666   Getting requirements to build wheel: finished with status 'done'
8.667   Preparing metadata (pyproject.toml): started
8.853   Preparing metadata (pyproject.toml): finished with status 'done'
8.899 Collecting pystac>=0.5.4 (from rio-tiler<7.0,>=6.3.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
8.923   Downloading pystac-1.10.1-py3-none-any.whl.metadata (6.4 kB)
8.961 Collecting affine (from rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
8.979   Downloading affine-2.4.0-py3-none-any.whl.metadata (4.0 kB)
9.016 Collecting certifi (from rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.036   Downloading certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
9.070 Collecting click>=4.0 (from rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.088   Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
9.119 Collecting cligj>=0.5 (from rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.137   Downloading cligj-0.7.2-py3-none-any.whl.metadata (5.0 kB)
9.173 Collecting snuggs>=1.4.1 (from rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.191   Downloading snuggs-1.4.7-py3-none-any.whl.metadata (3.4 kB)
9.218 Collecting click-plugins (from rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.239   Downloading click_plugins-1.1.1-py2.py3-none-any.whl.metadata (6.4 kB)
9.385 Collecting setuptools (from rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.386   Using cached setuptools-72.2.0-py3-none-any.whl.metadata (6.6 kB)
9.434 Collecting python-dateutil>=2.7.0 (from pystac>=0.5.4->rio-tiler<7.0,>=6.3.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.452   Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
9.499 Collecting pyparsing>=2.1.6 (from snuggs>=1.4.1->rasterio->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.526   Downloading pyparsing-3.1.2-py3-none-any.whl.metadata (5.1 kB)
9.560 Collecting anyio<5,>=3.4.0 (from starlette<0.39.0,>=0.37.2->fastapi>=0.107.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.579   Downloading anyio-4.4.0-py3-none-any.whl.metadata (4.6 kB)
9.619 Collecting httpcore==1.* (from httpx->cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.637   Downloading httpcore-1.0.5-py3-none-any.whl.metadata (20 kB)
9.671 Collecting idna (from httpx->cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.698   Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB)
9.728 Collecting sniffio (from httpx->cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.746   Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
9.783 Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx->cogeo-mosaic<8.0,>=7.0->titiler.mosaic<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.813   Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)
9.866 Collecting six>=1.5 (from python-dateutil>=2.7.0->pystac>=0.5.4->rio-tiler<7.0,>=6.3.0->titiler.core<0.18,>=0.17.0->titiler.pgstac==1.2.2->-r requirements.txt (line 1))
9.893   Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
9.947 Downloading titiler.pgstac-1.2.2-py3-none-any.whl (36 kB)
9.975 Downloading mangum-0.14.1-py3-none-any.whl (17 kB)
10.000 Downloading psycopg_binary-3.2.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.2 MB)
10.31    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 13.6 MB/s eta 0:00:00
10.33 Downloading geojson_pydantic-1.1.0-py3-none-any.whl (8.6 kB)
10.36 Downloading pydantic-2.8.2-py3-none-any.whl (423 kB)
10.41 Downloading pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
10.56    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 13.7 MB/s eta 0:00:00
10.58 Downloading pydantic_settings-2.4.0-py3-none-any.whl (23 kB)
10.64 Downloading titiler.core-0.17.3-py3-none-any.whl (32 kB)
10.70 Downloading titiler.mosaic-0.17.3-py3-none-any.whl (8.1 kB)
10.72 Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
10.75 Downloading psycopg-3.2.1-py3-none-any.whl (197 kB)
10.79 Downloading psycopg_pool-3.2.2-py3-none-any.whl (38 kB)
10.82 Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB)
10.85 Downloading cogeo_mosaic-7.1.0-py3-none-any.whl (39 kB)
10.87 Downloading fastapi-0.112.1-py3-none-any.whl (93 kB)
10.90 Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
10.94 Downloading morecantile-5.3.1-py3-none-any.whl (49 kB)
10.97 Downloading python_dotenv-1.0.1-py3-none-any.whl (19 kB)
10.99 Downloading rio_tiler-6.6.1-py3-none-any.whl (263 kB)
11.03 Downloading rasterio-1.3.10-cp311-cp311-manylinux2014_x86_64.whl (21.5 MB)
12.50    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.5/21.5 MB 14.7 MB/s eta 0:00:00
12.50 Using cached numpy-2.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.3 MB)
12.54 Downloading simplejson-3.19.3-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (144 kB)
12.58 Downloading click-8.1.7-py3-none-any.whl (97 kB)
12.62 Downloading cligj-0.7.2-py3-none-any.whl (7.1 kB)
12.64 Downloading MarkupSafe-2.1.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
12.67 Downloading pyproj-3.6.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.6 MB)
13.27    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.6/8.6 MB 14.2 MB/s eta 0:00:00
13.29 Downloading pystac-1.10.1-py3-none-any.whl (182 kB)
13.33 Downloading shapely-2.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
13.52    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 12.9 MB/s eta 0:00:00
13.54 Downloading snuggs-1.4.7-py3-none-any.whl (5.4 kB)
13.57 Downloading starlette-0.38.2-py3-none-any.whl (72 kB)
13.60 Downloading affine-2.4.0-py3-none-any.whl (15 kB)
13.63 Downloading attrs-24.2.0-py3-none-any.whl (63 kB)
13.65 Downloading cachetools-5.5.0-py3-none-any.whl (9.5 kB)
13.68 Downloading certifi-2024.7.4-py3-none-any.whl (162 kB)
13.72 Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB)
13.74 Downloading color_operations-0.1.5-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)
13.79 Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
13.82 Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
13.83 Using cached setuptools-72.2.0-py3-none-any.whl (2.3 MB)
13.85 Downloading supermorecado-0.1.2-py3-none-any.whl (14 kB)
13.88 Downloading anyio-4.4.0-py3-none-any.whl (86 kB)
13.91 Downloading idna-3.7-py3-none-any.whl (66 kB)
13.94 Downloading pyparsing-3.1.2-py3-none-any.whl (103 kB)
13.96 Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
14.01 Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
14.03 Downloading h11-0.14.0-py3-none-any.whl (58 kB)
14.06 Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
14.16 Building wheels for collected packages: numexpr
14.17   Building wheel for numexpr (pyproject.toml): started
14.52   Building wheel for numexpr (pyproject.toml): finished with status 'error'
14.53   error: subprocess-exited-with-error
14.53
14.53   × Building wheel for numexpr (pyproject.toml) did not run successfully.
14.53   │ exit code: 1
14.53   ╰─> [136 lines of output]
14.53       /tmp/pip-build-env-gz5ktou9/overlay/lib/python3.11/site-packages/setuptools/config/_apply_pyprojecttoml.py:79: SetuptoolsWarning: `install_requires` overwritten in `pyproject.toml` (dependencies)
14.53         corresp(dist, value, root_dir)
14.53       running bdist_wheel
14.53       running build
14.53       running build_py
14.53       creating build
14.53       creating build/lib.linux-x86_64-cpython-311
14.53       creating build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/__init__.py -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/cpuinfo.py -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/version.py -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/expressions.py -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/utils.py -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/necompiler.py -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       running egg_info
14.53       writing numexpr.egg-info/PKG-INFO
14.53       writing dependency_links to numexpr.egg-info/dependency_links.txt
14.53       writing requirements to numexpr.egg-info/requires.txt
14.53       writing top-level names to numexpr.egg-info/top_level.txt
14.53       reading manifest file 'numexpr.egg-info/SOURCES.txt'
14.53       reading manifest template 'MANIFEST.in'
14.53       warning: no previously-included files found matching 'numexpr/__config__.py'
14.53       warning: no previously-included files found matching 'RELEASING.txt'
14.53       warning: no previously-included files found matching 'site.cfg'
14.53       adding license file 'LICENSE.txt'
14.53       adding license file 'AUTHORS.txt'
14.53       writing manifest file 'numexpr.egg-info/SOURCES.txt'
14.53       /tmp/pip-build-env-gz5ktou9/overlay/lib/python3.11/site-packages/setuptools/command/build_py.py:215: _Warning: Package 'numexpr.tests' is absent from the `packages` configuration.
14.53       !!
14.53
14.53               ********************************************************************************
14.53               ############################
14.53               # Package would be ignored #
14.53               ############################
14.53               Python recognizes 'numexpr.tests' as an importable package[^1],
14.53               but it is absent from setuptools' `packages` configuration.
14.53
14.53               This leads to an ambiguous overall configuration. If you want to distribute this
14.53               package, please make sure that 'numexpr.tests' is explicitly added
14.53               to the `packages` configuration field.
14.53
14.53               Alternatively, you can also rely on setuptools' discovery methods
14.53               (for example by using `find_namespace_packages(...)`/`find_namespace:`
14.53               instead of `find_packages(...)`/`find:`).
14.53
14.53               You can read more about "package discovery" on setuptools documentation page:
14.53
14.53               - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
14.53
14.53               If you don't want 'numexpr.tests' to be distributed and are
14.53               already explicitly excluding 'numexpr.tests' via
14.53               `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
14.53               you can try to use `exclude_package_data`, or `include-package-data=False` in
14.53               combination with a more fine grained `package-data` configuration.
14.53
14.53               You can read more about "package data files" on setuptools documentation page:
14.53
14.53               - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
14.53
14.53
14.53               [^1]: For Python, any directory (with suitable naming) can be imported,
14.53                     even if it does not contain any `.py` files.
14.53                     On the other hand, currently there is no concept of package data
14.53                     directory, all directories are treated like packages.
14.53               ********************************************************************************
14.53
14.53       !!
14.53         check.warn(importable)
14.53       /tmp/pip-build-env-gz5ktou9/overlay/lib/python3.11/site-packages/setuptools/command/build_py.py:215: _Warning: Package 'numexpr.win32' is absent from the `packages` configuration.
14.53       !!
14.53
14.53               ********************************************************************************
14.53               ############################
14.53               # Package would be ignored #
14.53               ############################
14.53               Python recognizes 'numexpr.win32' as an importable package[^1],
14.53               but it is absent from setuptools' `packages` configuration.
14.53
14.53               This leads to an ambiguous overall configuration. If you want to distribute this
14.53               package, please make sure that 'numexpr.win32' is explicitly added
14.53               to the `packages` configuration field.
14.53
14.53               Alternatively, you can also rely on setuptools' discovery methods
14.53               (for example by using `find_namespace_packages(...)`/`find_namespace:`
14.53               instead of `find_packages(...)`/`find:`).
14.53
14.53               You can read more about "package discovery" on setuptools documentation page:
14.53
14.53               - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
14.53
14.53               If you don't want 'numexpr.win32' to be distributed and are
14.53               already explicitly excluding 'numexpr.win32' via
14.53               `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
14.53               you can try to use `exclude_package_data`, or `include-package-data=False` in
14.53               combination with a more fine grained `package-data` configuration.
14.53
14.53               You can read more about "package data files" on setuptools documentation page:
14.53
14.53               - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
14.53
14.53
14.53               [^1]: For Python, any directory (with suitable naming) can be imported,
14.53                     even if it does not contain any `.py` files.
14.53                     On the other hand, currently there is no concept of package data
14.53                     directory, all directories are treated like packages.
14.53               ********************************************************************************
14.53
14.53       !!
14.53         check.warn(importable)
14.53       copying numexpr/complex_functions.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/functions.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/interp_body.cpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/interpreter.cpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/interpreter.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/missing_posix_functions.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/module.cpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/module.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/msvc_function_stubs.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/numexpr_config.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/numexpr_object.cpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/numexpr_object.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/opcodes.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       copying numexpr/str-two-way.hpp -> build/lib.linux-x86_64-cpython-311/numexpr
14.53       creating build/lib.linux-x86_64-cpython-311/numexpr/tests
14.53       copying numexpr/tests/__init__.py -> build/lib.linux-x86_64-cpython-311/numexpr/tests
14.53       copying numexpr/tests/test_numexpr.py -> build/lib.linux-x86_64-cpython-311/numexpr/tests
14.53       creating build/lib.linux-x86_64-cpython-311/numexpr/win32
14.53       copying numexpr/win32/pthread.c -> build/lib.linux-x86_64-cpython-311/numexpr/win32
14.53       copying numexpr/win32/pthread.h -> build/lib.linux-x86_64-cpython-311/numexpr/win32
14.53       copying numexpr/win32/stdint.h -> build/lib.linux-x86_64-cpython-311/numexpr/win32
14.53       running build_ext
14.53       building 'numexpr.interpreter' extension
14.53       creating build/temp.linux-x86_64-cpython-311
14.53       creating build/temp.linux-x86_64-cpython-311/numexpr
14.53       g++ -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DNPY_TARGET_VERSION=NPY_1_23_API_VERSION -I/tmp/pip-build-env-gz5ktou9/overlay/lib/python3.11/site-packages/numpy/_core/include -I/var/lang/include/python3.11 -c numexpr/interpreter.cpp -o build/temp.linux-x86_64-cpython-311/numexpr/interpreter.o
14.53       error: command 'g++' failed: No such file or directory
14.53       [end of output]
14.53
14.53   note: This error originates from a subprocess, and is likely not a problem with pip.
14.53   ERROR: Failed building wheel for numexpr
14.53 Failed to build numexpr
14.54 ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (numexpr)
------

 2 warnings found (use docker --debug to expand):
 - InvalidDefaultArgInFrom: Default value for ARG public.ecr.aws/lambda/python:${PYTHON_VERSION} results in empty or invalid base image name (line 2)
 - FromPlatformFlagConstDisallowed: FROM --platform flag should not use constant value "linux/amd64" (line 2)
Dockerfile:8
--------------------
   6 |
   7 |     COPY runtime/requirements.txt requirements.txt
   8 | >>> RUN python -m pip install -r requirements.txt "mangum>=0.14,<0.15" -t /asset
   9 |
  10 |     # Reduce package size and remove useless files
--------------------
ERROR: failed to solve: process "/bin/sh -c python -m pip install -r requirements.txt \"mangum>=0.14,<0.15\" -t /asset" did not complete successfully: exit code: 1
bransonf commented 2 months ago

Suggested a straightforward, minimal fix in #108 and will leave it to you all to determine if this is appropriate.

vincentsarago commented 2 months ago

FYI this is related to https://github.com/pydata/numexpr/issues/493 https://github.com/developmentseed/titiler/issues/946

hrodmn commented 2 months ago

I tested two solutions:

Since the CDK is only uploading the /asset directory from the docker image build, adding the g++ installation has no effect on the code package size (it is 61 MB either way).

It's fine with me if we want to just follow @vincentsarago's approach and pin numexpr to <2.10.1 but it should also be ok to install the compiler in the docker image in this case.

vincentsarago commented 2 months ago

🙏 thanks @hrodmn, I think it's a much better solution to add g++ installation 🙏