search

ajnisbet / opentopodata

Open alternative to the Google Elevation API!
https://www.opentopodata.org
MIT License
326 stars 71 forks source link

WARNING:root:Failed to get options via gdal-config (M1 mac) #55

Closed HugoHeneault closed 3 years ago

HugoHeneault commented 3 years ago

Hi there,

I'm running make build on macOS (m1) 12.0.1 and encounter this issue:

#11 6.510 Collecting rasterio==1.2.9
#11 6.553   Downloading rasterio-1.2.9.tar.gz (2.3 MB)
#11 7.162   Installing build dependencies: started
#11 13.24   Installing build dependencies: finished with status 'done'
#11 13.24   Getting requirements to build wheel: started
#11 13.45   Getting requirements to build wheel: finished with status 'error'
#11 13.45   ERROR: Command errored out with exit status 1:
#11 13.45    command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_
#11 13.45        cwd: /tmp/pip-install-rusm7ufj/rasterio_d6e3703bd238492fb428a60f0b4d1a47
#11 13.45   Complete output (2 lines):
#11 13.45   WARNING:root:Failed to get options via gdal-config: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
#11 13.45   ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.
#11 13.45   ----------------------------------------
#11 13.45 WARNING: Discarding https://files.pythonhosted.org/packages/1f/70/4e7a789f4988955e4c381de80923e184f912683bbe6fc4a3a00c91efdf59/rasterio-1.2.9.tar.gz#sha256=012a4964d8db365be4fae0af9cbeba00e683e5904d5031e8ba42ccb6040cc887 (from https://pypi.org/simple/rasterio/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_ Check the logs for full command output.
#11 13.45 ERROR: Could not find a version that satisfies the requirement rasterio==1.2.9 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.5.1, 0.6, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.9, 0.10, 0.10.1, 0.11, 0.11.1, 0.12, 0.12.1, 0.13, 0.13.1, 0.13.2, 0.14, 0.14.1, 0.15, 0.15.1, 0.16, 0.17, 0.17.1, 0.18, 0.19.0, 0.19.1, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.24.1, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.0.post1, 0.33.0, 0.34.0, 0.35.0, 0.35.0.post1, 0.35.1, 0.36.0, 1.0a1, 1.0a2, 1.0a3, 1.0a4, 1.0a6, 1.0a7, 1.0a8, 1.0a9, 1.0a10, 1.0a11, 1.0a12, 1.0b1, 1.0b2, 1.0b3, 1.0b4, 1.0rc1, 1.0rc2, 1.0rc3, 1.0rc4, 1.0rc5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.3.post1, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10, 1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.0.16, 1.0.17, 1.0.18, 1.0.20, 1.0.21, 1.0.22, 1.0.23, 1.0.24, 1.0.25, 1.0.26, 1.0.27, 1.0.28, 1.1b1, 1.1b2, 1.1b3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.2a1, 1.2b1, 1.2b2, 1.2b3, 1.2b4, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.3a1, 1.3a2)
#11 13.46 ERROR: No matching distribution found for rasterio==1.2.9
------
executor failed running [/bin/sh -c pip install         --no-index         --no-cache-dir         --disable-pip-version-check         --find-links=/root/wheels         uwsgi regex &&     pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt &&         rm -rf /root/.cache/pip/* &&         rm root/wheels/* &&         rm /app/requirements.txt]: exit code: 1
make: *** [build] Error 1
Full logs ``` $ make build docker build --tag opentopodata:`cat VERSION` --file docker/Dockerfile . [+] Building 18.1s (11/14) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 37B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [internal] load metadata for docker.io/library/python:3.7.12-slim-buster 4.3s => [internal] load build context 0.0s => => transferring context: 5.39kB 0.0s => [builder 1/3] FROM docker.io/library/python:3.7.12-slim-buster@sha256:866b51113519f9022e2f432869d3e7d97b090b9853f4765dbf46394ac28d5a03 0.0s => CACHED [stage-1 2/8] RUN set -e && apt-get update && apt-get install -y --no-install-recommends nginx memcached supervisor && rm -rf /var/lib/apt/lists/* 0.0s => CACHED [builder 2/3] RUN set -e && apt-get update && apt-get install -y --no-install-recommends gcc python3.7-dev 0.0s => CACHED [builder 3/3] RUN pip wheel --wheel-dir=/root/wheels uwsgi==2.0.19.1 && pip wheel --wheel-dir=/root/wheels regex==2021.9.30 0.0s => CACHED [stage-1 3/8] COPY --from=builder /root/wheels /root/wheels 0.0s => CACHED [stage-1 4/8] COPY requirements.txt /app/requirements.txt 0.0s => ERROR [stage-1 5/8] RUN pip install --no-index --no-cache-dir --disable-pip-version-check --find-links=/root/wheels uwsgi regex && pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt && rm -rf /root/.cache/pip/* && rm root/wheels/* && rm /app/requirements.txt 13.6s ------ > [stage-1 5/8] RUN pip install --no-index --no-cache-dir --disable-pip-version-check --find-links=/root/wheels uwsgi regex && pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt && rm -rf /root/.cache/pip/* && rm root/wheels/* && rm /app/requirements.txt: #11 1.033 Looking in links: /root/wheels #11 1.040 Processing /root/wheels/uWSGI-2.0.19.1-cp37-cp37m-linux_aarch64.whl #11 1.044 Processing /root/wheels/regex-2021.9.30-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl #11 1.066 Installing collected packages: uwsgi, regex #11 1.157 Successfully installed regex-2021.9.30 uwsgi-2.0.19.1 #11 1.157 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv #11 1.536 Collecting affine==2.3.0 #11 1.706 Downloading affine-2.3.0-py2.py3-none-any.whl (15 kB) #11 1.746 Collecting attrs==21.2.0 #11 1.773 Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB) #11 1.839 Collecting black==21.9b0 #11 1.868 Downloading black-21.9b0-py3-none-any.whl (148 kB) #11 1.944 Collecting certifi==2021.5.30 #11 1.980 Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB) #11 2.051 Collecting charset-normalizer==2.0.6 #11 2.082 Downloading charset_normalizer-2.0.6-py3-none-any.whl (37 kB) #11 2.121 Collecting click==8.0.1 #11 2.147 Downloading click-8.0.1-py3-none-any.whl (97 kB) #11 2.189 Collecting click-plugins==1.1.1 #11 2.215 Downloading click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB) #11 2.266 Collecting cligj==0.7.2 #11 2.290 Downloading cligj-0.7.2-py3-none-any.whl (7.1 kB) #11 2.632 Collecting coverage[toml]==6.0.1 #11 2.661 Downloading coverage-6.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (243 kB) #11 2.726 Collecting flask==2.0.2 #11 2.764 Downloading Flask-2.0.2-py3-none-any.whl (95 kB) #11 2.811 Collecting flask-caching==1.10.1 #11 2.836 Downloading Flask_Caching-1.10.1-py3-none-any.whl (34 kB) #11 2.873 Collecting geographiclib==1.52 #11 2.898 Downloading geographiclib-1.52-py3-none-any.whl (38 kB) #11 2.938 Collecting idna==3.2 #11 2.964 Downloading idna-3.2-py3-none-any.whl (59 kB) #11 3.027 Collecting importlib-metadata==4.8.1 #11 3.052 Downloading importlib_metadata-4.8.1-py3-none-any.whl (17 kB) #11 3.083 Collecting iniconfig==1.1.1 #11 3.128 Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB) #11 3.160 Collecting itsdangerous==2.0.1 #11 3.200 Downloading itsdangerous-2.0.1-py3-none-any.whl (18 kB) #11 3.241 Collecting jinja2==3.0.2 #11 3.282 Downloading Jinja2-3.0.2-py3-none-any.whl (133 kB) #11 3.361 Collecting markupsafe==2.0.1 #11 3.384 Downloading MarkupSafe-2.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB) #11 3.409 Collecting mypy-extensions==0.4.3 #11 3.441 Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB) #11 3.736 Collecting numpy==1.21.2 #11 3.770 Downloading numpy-1.21.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.0 MB) #11 4.474 Collecting packaging==21.0 #11 4.500 Downloading packaging-21.0-py3-none-any.whl (40 kB) #11 4.544 Collecting pathspec==0.9.0 #11 4.575 Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB) #11 4.606 Collecting pep517==0.11.0 #11 4.640 Downloading pep517-0.11.0-py2.py3-none-any.whl (19 kB) #11 4.702 Collecting pip-tools==6.3.0 #11 4.727 Downloading pip_tools-6.3.0-py3-none-any.whl (47 kB) #11 4.762 Collecting platformdirs==2.4.0 #11 4.793 Downloading platformdirs-2.4.0-py3-none-any.whl (14 kB) #11 4.843 Collecting pluggy==1.0.0 #11 4.871 Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB) #11 4.898 Collecting polyline==1.4.0 #11 4.934 Downloading polyline-1.4.0-py2.py3-none-any.whl (4.4 kB) #11 4.986 Collecting py==1.10.0 #11 5.015 Downloading py-1.10.0-py2.py3-none-any.whl (97 kB) #11 5.055 Collecting pylibmc==1.6.1 #11 5.081 Downloading pylibmc-1.6.1.tar.gz (64 kB) #11 5.651 Collecting pyparsing==2.4.7 #11 5.684 Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) #11 5.785 Collecting pyproj==3.2.1 #11 5.822 Downloading pyproj-3.2.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (6.1 MB) #11 6.103 Collecting pytest==6.2.5 #11 6.146 Downloading pytest-6.2.5-py3-none-any.whl (280 kB) #11 6.205 Collecting pytest-cov==3.0.0 #11 6.243 Downloading pytest_cov-3.0.0-py3-none-any.whl (20 kB) #11 6.313 Collecting pyyaml==5.4.1 #11 6.347 Downloading PyYAML-5.4.1-cp37-cp37m-manylinux2014_aarch64.whl (716 kB) #11 6.510 Collecting rasterio==1.2.9 #11 6.553 Downloading rasterio-1.2.9.tar.gz (2.3 MB) #11 7.162 Installing build dependencies: started #11 13.24 Installing build dependencies: finished with status 'done' #11 13.24 Getting requirements to build wheel: started #11 13.45 Getting requirements to build wheel: finished with status 'error' #11 13.45 ERROR: Command errored out with exit status 1: #11 13.45 command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_ #11 13.45 cwd: /tmp/pip-install-rusm7ufj/rasterio_d6e3703bd238492fb428a60f0b4d1a47 #11 13.45 Complete output (2 lines): #11 13.45 WARNING:root:Failed to get options via gdal-config: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config' #11 13.45 ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable. #11 13.45 ---------------------------------------- #11 13.45 WARNING: Discarding https://files.pythonhosted.org/packages/1f/70/4e7a789f4988955e4c381de80923e184f912683bbe6fc4a3a00c91efdf59/rasterio-1.2.9.tar.gz#sha256=012a4964d8db365be4fae0af9cbeba00e683e5904d5031e8ba42ccb6040cc887 (from https://pypi.org/simple/rasterio/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_b8p17k_ Check the logs for full command output. #11 13.45 ERROR: Could not find a version that satisfies the requirement rasterio==1.2.9 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.5.1, 0.6, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.9, 0.10, 0.10.1, 0.11, 0.11.1, 0.12, 0.12.1, 0.13, 0.13.1, 0.13.2, 0.14, 0.14.1, 0.15, 0.15.1, 0.16, 0.17, 0.17.1, 0.18, 0.19.0, 0.19.1, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.24.1, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.0.post1, 0.33.0, 0.34.0, 0.35.0, 0.35.0.post1, 0.35.1, 0.36.0, 1.0a1, 1.0a2, 1.0a3, 1.0a4, 1.0a6, 1.0a7, 1.0a8, 1.0a9, 1.0a10, 1.0a11, 1.0a12, 1.0b1, 1.0b2, 1.0b3, 1.0b4, 1.0rc1, 1.0rc2, 1.0rc3, 1.0rc4, 1.0rc5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.3.post1, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10, 1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.0.16, 1.0.17, 1.0.18, 1.0.20, 1.0.21, 1.0.22, 1.0.23, 1.0.24, 1.0.25, 1.0.26, 1.0.27, 1.0.28, 1.1b1, 1.1b2, 1.1b3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.2a1, 1.2b1, 1.2b2, 1.2b3, 1.2b4, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.3a1, 1.3a2) #11 13.46 ERROR: No matching distribution found for rasterio==1.2.9 ------ executor failed running [/bin/sh -c pip install --no-index --no-cache-dir --disable-pip-version-check --find-links=/root/wheels uwsgi regex && pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt && rm -rf /root/.cache/pip/* && rm root/wheels/* && rm /app/requirements.txt]: exit code: 1 make: *** [build] Error 1 ```

Any idea of what's going on? How could I help you help me fix this? ;-)

Thanks!

ajnisbet commented 3 years ago

Hmm, this is strange: it's trying to build rasterio from source, instead of just downloading the wheel. This will fail as the docker image doesn't have the dependencies (like gdal) to build rasterio.

There is a wheel for rasterio 1.2.9 so that should get preferred. When I run the build from scratch, I get 

...
Collecting pytest-cov==3.0.0
  Downloading pytest_cov-3.0.0-py3-none-any.whl (20 kB)
Collecting pyyaml==5.4.1
  Downloading PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636 kB)
Collecting rasterio==1.2.9
  Downloading rasterio-1.2.9-cp37-cp37m-manylinux1_x86_64.whl (19.3 MB)
Requirement already satisfied: regex==2021.9.30 in /usr/local/lib/python3.7/site-packages (from -r /app/requirements.txt (line 101)) (2021.9.30)
Collecting requests==2.26.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting six==1.16.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting snuggs==1.4.7
...

A workaround would be to download the wheel first then install it manually.

But first could you try building with docker build --tag opentopodata:1.7.0 --file docker/Dockerfile --no-cache .

This will force rebuilding of all layers, perhaps some bad data got cached.

HugoHeneault commented 3 years ago

Thanks for your extra-fast reply!

But I'm still having the same issue. This might be due to macOS m1 architecture?

I tried adding the wheel by adding pip wheel --wheel-dir=/root/wheels rasterio==1.2.9 but it is also failing.

HugoHeneault commented 3 years ago

Here's the error I have when adding wheel with pip wheel:

``` make build docker build --tag opentopodata:`cat VERSION` --file docker/Dockerfile . [+] Building 8.7s (9/15) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 37B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => [internal] load metadata for docker.io/library/python:3.7.12-slim-buster 1.2s => [builder 1/3] FROM docker.io/library/python:3.7.12-slim-buster@sha256:866b51113519f9022e2f432869d3e7d97b090b9853f4765dbf46394ac28d5a03 0.0s => [internal] load build context 0.0s => => transferring context: 17.73kB 0.0s => CACHED [builder 2/3] RUN set -e && apt-get update && apt-get install -y --no-install-recommends gcc python3.7-dev 0.0s => CANCELED [builder 3/3] RUN pip wheel --wheel-dir=/root/wheels uwsgi==2.0.19.1 && pip wheel --wheel-dir=/root/wheels regex==2021.9.30 && pip wheel --wheel-dir=/root/wheels rasterio==1.2.9 7.4s => CACHED [stage-1 2/9] RUN set -e && apt-get update && apt-get install -y --no-install-recommends nginx memcached supervisor && rm -rf /var/lib/apt/lists/* 0.0s => ERROR [stage-1 3/9] RUN pip wheel --wheel-dir=/root/wheels rasterio==1.2.9 7.4s ------ > [stage-1 3/9] RUN pip wheel --wheel-dir=/root/wheels rasterio==1.2.9: #6 1.252 Collecting rasterio==1.2.9 #6 1.357 Downloading rasterio-1.2.9.tar.gz (2.3 MB) #6 2.014 Installing build dependencies: started #6 6.754 Installing build dependencies: finished with status 'done' #6 6.762 Getting requirements to build wheel: started #6 7.195 Getting requirements to build wheel: finished with status 'error' #6 7.195 ERROR: Command errored out with exit status 1: #6 7.195 command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_m9xu3wj #6 7.195 cwd: /tmp/pip-wheel-2xm3n6f8/rasterio_94950f13d5dc46e4ba3635ec82daf4b5 #6 7.195 Complete output (2 lines): #6 7.195 WARNING:root:Failed to get options via gdal-config: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config' #6 7.195 ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable. #6 7.195 ---------------------------------------- #6 7.196 WARNING: Discarding https://files.pythonhosted.org/packages/1f/70/4e7a789f4988955e4c381de80923e184f912683bbe6fc4a3a00c91efdf59/rasterio-1.2.9.tar.gz#sha256=012a4964d8db365be4fae0af9cbeba00e683e5904d5031e8ba42ccb6040cc887 (from https://pypi.org/simple/rasterio/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp_m9xu3wj Check the logs for full command output. #6 7.197 ERROR: Could not find a version that satisfies the requirement rasterio==1.2.9 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.5.1, 0.6, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.9, 0.10, 0.10.1, 0.11, 0.11.1, 0.12, 0.12.1, 0.13, 0.13.1, 0.13.2, 0.14, 0.14.1, 0.15, 0.15.1, 0.16, 0.17, 0.17.1, 0.18, 0.19.0, 0.19.1, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.24.1, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.0.post1, 0.33.0, 0.34.0, 0.35.0, 0.35.0.post1, 0.35.1, 0.36.0, 1.0a1, 1.0a2, 1.0a3, 1.0a4, 1.0a6, 1.0a7, 1.0a8, 1.0a9, 1.0a10, 1.0a11, 1.0a12, 1.0b1, 1.0b2, 1.0b3, 1.0b4, 1.0rc1, 1.0rc2, 1.0rc3, 1.0rc4, 1.0rc5, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.3.post1, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10, 1.0.11, 1.0.12, 1.0.13, 1.0.14, 1.0.15, 1.0.16, 1.0.17, 1.0.18, 1.0.20, 1.0.21, 1.0.22, 1.0.23, 1.0.24, 1.0.25, 1.0.26, 1.0.27, 1.0.28, 1.1b1, 1.1b2, 1.1b3, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.2a1, 1.2b1, 1.2b2, 1.2b3, 1.2b4, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.3a1, 1.3a2) #6 7.198 ERROR: No matching distribution found for rasterio==1.2.9 #6 7.204 WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available. #6 7.204 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. ------ executor failed running [/bin/sh -c pip wheel --wheel-dir=/root/wheels rasterio==1.2.9]: exit code: 1 make: *** [build] Error 1 ```
HugoHeneault commented 3 years ago

I also tried running make build in rosetta mode, still the same issue.

HugoHeneault commented 3 years ago

I'm now looking for a way to build gdal which would allow us to build rasterio from sources...

ajnisbet commented 3 years ago

Ah, I missed that you're on an M1 mac, I imagine that is the issue.

I don't have a macbook to test on, and I don't quite understand how M1, docker and rosetta all work together. I'm surprised rosetta didn't work.

I know rasterio doesn't have wheels for M1s (https://github.com/rasterio/rasterio-wheels/issues/69). Even on linux installing gdal and proj is a headache, which is why I went the rasterio route.

This Dockerfile works for me on linux (it builds and the tests pass): on my linux machine it pulls the rasterio wheel of course, but if the base image works on m1 that should have the dependencies needed to build rasterio

FROM osgeo/gdal:ubuntu-full-3.4.0
RUN set -e && \
    apt-get update && \
    apt-get install -y --no-install-recommends \
        nginx \
        memcached \
        python3-pip \
        gcc \
        g++ \
        supervisor \
        libmemcached-dev \
        python3.8-dev && \
    rm -rf /var/lib/apt/lists/*

COPY requirements.txt /app/requirements.txt
RUN pip install \
        --no-cache-dir \
        --disable-pip-version-check \
        uwsgi regex && \
    pip install --no-cache-dir --disable-pip-version-check -r /app/requirements.txt && \
        rm -rf /root/.cache/pip/* && \
        rm /app/requirements.txt

WORKDIR /app
COPY . /app/

RUN echo > /etc/nginx/sites-available/default && \
    cp /app/docker/nginx.conf /etc/nginx/conf.d/nginx.conf && \
    cp /app/docker/supervisord.conf /etc/supervisor/conf.d/supervisord.conf

CMD ["sh", "/app/docker/run.sh"]
EXPOSE 5000

ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
ENV GDAL_DISABLE_READDIR_ON_OPEN=TRUE
ENV GDAL_NUM_THREADS=ALL_CPUS
ENV GDAL_CACHEMAX=512

Otherwise you could try running without docker using homebrew and/or conda.

HugoHeneault commented 3 years ago

This Dockerfile script had ended without error. Awesome! Thanks a lot for your help! 🎉

I'll report back of my m1 experience.

I just read this article - which might need a bit of updates. Hope I won't get into nightmares on other projects. 🤷‍♂️

ajnisbet commented 3 years ago

Ok fantastic! The one I posted above? Let me know if you have any issues, otherwise I can add that file as M1.Dockerfile for others.

HugoHeneault commented 3 years ago

I had to change one more thing to get it working. I think python's path is different so I simplified it:

I replaced command=/usr/local/bin/python /app/docker/warm_cache.py by command=python /app/docker/warm_cache.py: 

[program:warm_cache]
user=www-data
command=python /app/docker/warm_cache.py
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
autorestart=false

Server is now running fine and I get elevation data 👏

Do you want me to add a note on README for M1 owners & do the supervisor conf change?

Thanks again!

HugoHeneault commented 3 years ago

I can also confirm that using your Dockerfile posted above works without Rosetta. 🎉

ajnisbet commented 3 years ago

@HugoHeneault could you make PR for the supervisor conf change? I can confirm that works with the standard Dockerfile too.

I'll update the docs.

Thank you!

HugoHeneault commented 3 years ago

@ajnisbet Yup just did it in #56 :-)

Have a great day!

ajnisbet commented 3 years ago

Thanks for your help!