search

openfaas / python-flask-template

HTTP and Flask-based OpenFaaS templates for Python 3
MIT License
85 stars 86 forks source link

Build fails in python3-flask template #32

Closed martindekov closed 4 years ago

martindekov commented 4 years ago

Decription

When trying to build function with the: python3-flask template the 15 step fails with this error:

ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-zh4yzal0/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 40.8.0' wheel 'Cython >= 0.29.14' 'cffi >= 1.12.3 ; platform_python_implementation == '"'"'CPython'"'"'' 'greenlet>=0.4.14 ; platform_python_implementation == '"'"'CPython'"'"'' Check the logs for full command output.
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

The step which fails is:

RUN pip install -r requirements.txt

which installs:

flask
gevent

I see flask succeeds being fetched, but gevent fails.

Full logs

The full logs are the following:

Step 15/32 : RUN pip install -r requirements.txt
 ---> Running in 5bcfc8dc5f75
Collecting flask
  Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting gevent
  Downloading gevent-20.4.0.tar.gz (5.5 MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-zh4yzal0/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 40.8.0' wheel 'Cython >= 0.29.14' 'cffi >= 1.12.3 ; platform_python_implementation == '"'"'CPython'"'"'' 'greenlet>=0.4.14 ; platform_python_implementation == '"'"'CPython'"'"''
       cwd: None
  Complete output (109 lines):
  Collecting setuptools>=40.8.0
    Downloading setuptools-46.1.3-py3-none-any.whl (582 kB)
  Collecting wheel
    Downloading wheel-0.34.2-py2.py3-none-any.whl (26 kB)
  Collecting Cython>=0.29.14
    Downloading Cython-0.29.17-py2.py3-none-any.whl (971 kB)
  Collecting cffi>=1.12.3
    Downloading cffi-1.14.0.tar.gz (463 kB)
  Collecting greenlet>=0.4.14
    Downloading greenlet-0.4.15.tar.gz (59 kB)
  Collecting pycparser
    Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
  Building wheels for collected packages: cffi, greenlet
    Building wheel for cffi (setup.py): started
    Building wheel for cffi (setup.py): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u0umok24/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u0umok24/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ehri8a3m
         cwd: /tmp/pip-install-u0umok24/cffi/
    Complete output (36 lines):
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/cffi
    copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/verifier.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/recompiler.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/api.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/__init__.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/commontypes.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/cparser.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/lock.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/error.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/model.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/_embedding.h -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.7/cffi
    running build_ext
    building '_cffi_backend' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/c
    gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
    c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
       15 | #include <ffi.h>
          |          ^~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
    ERROR: Failed building wheel for cffi
    Running setup.py clean for cffi
    Building wheel for greenlet (setup.py): started
    Building wheel for greenlet (setup.py): finished with status 'done'
    Created wheel for greenlet: filename=greenlet-0.4.15-cp37-cp37m-linux_x86_64.whl size=50272 sha256=61a30c0689692a1cdcc99cc917c3c9426baa1e6c5c33d1dbcba581799006d49b
    Stored in directory: /root/.cache/pip/wheels/9f/ef/08/f28669af76917de4c18abfd28a491c67c8ac3166d45d00660c
  Successfully built greenlet
  Failed to build cffi
  Installing collected packages: setuptools, wheel, Cython, pycparser, cffi, greenlet
      Running setup.py install for cffi: started
      Running setup.py install for cffi: finished with status 'error'
      ERROR: Command errored out with exit status 1:
       command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u0umok24/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u0umok24/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-rynut_o8/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-zh4yzal0/overlay --compile --install-headers /tmp/pip-build-env-zh4yzal0/overlay/include/python3.7m/cffi
           cwd: /tmp/pip-install-u0umok24/cffi/
      Complete output (36 lines):
      running install
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.7
      creating build/lib.linux-x86_64-3.7/cffi
      copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/verifier.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/recompiler.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/api.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/__init__.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/commontypes.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/cparser.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/lock.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/error.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/model.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/_embedding.h -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.7/cffi
      running build_ext
      building '_cffi_backend' extension
      creating build/temp.linux-x86_64-3.7
      creating build/temp.linux-x86_64-3.7/c
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
      c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
         15 | #include <ffi.h>
            |          ^~~~~~~
      compilation terminated.
      error: command 'gcc' failed with exit status 1
      ----------------------------------------
  ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-u0umok24/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-u0umok24/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-rynut_o8/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-zh4yzal0/overlay --compile --install-headers /tmp/pip-build-env-zh4yzal0/overlay/include/python3.7m/cffi Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-zh4yzal0/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 40.8.0' wheel 'Cython >= 0.29.14' 'cffi >= 1.12.3 ; platform_python_implementation == '"'"'CPython'"'"'' 'greenlet>=0.4.14 ; platform_python_implementation == '"'"'CPython'"'"'' Check the logs for full command output.
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

Steps to reproduce

  1. Pull function from store
  2. Build function with the template
  3. See the above failure
Jeff-Lowrey commented 4 years ago

@alexellis @LucasRoesler Some comments:

  1. This is reproducible (for me) using the steps provided
  2. There are a number of other places/projects having this same issue, including https://github.com/gliderlabs/docker-alpine/issues/297
  3. It's resolvable by adding libffi-dev to the apk command in the docker file RUN apk --no-cache add musl-dev libffi-dev gcc make ${ADDITIONAL_PACKAGE}

In addition, oddly, some investigation shows that the libs are installed in the base python:3.7-alpine image.

EDIT: These are the runtime libraries and not related to the dev/source files needed. Ooops.

Using python:3.7-alpine (image id 16f919b9ecd5)

docker run -it 16f919b9ecd5 sh

/ # ls -la /usr/lib
...
drwxr-xr-x    2 root     root          4096 Apr 24 22:03 libffi-3.2.1
lrwxrwxrwx    1 root     root            15 Apr 24 22:03 libffi.so.6 -> libffi.so.6.0.4
-rwxr-xr-x    1 root     root         30424 Nov 15 16:01 libffi.so.6.0.4
...

/ # whoami
root
/ # env|sort
GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568
HOME=/root
HOSTNAME=bde6d170a2dd
LANG=C.UTF-8
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
PYTHON_GET_PIP_SHA256=421ac1d44c0cf9730a088e337867d974b91bdce4ea2636099275071878cc189e
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/d59197a3c169cef378a22428a3fa99d33e080a5d/get-pip.py
PYTHON_PIP_VERSION=20.0.2
PYTHON_VERSION=3.8.2
SHLVL=1
TERM=xterm

I haven't done any real research to find out how new this issue is, but it does look to be some kind of issue with the Alpine release.

LucasRoesler commented 4 years ago

I can also confirm this and that @Jeff-Lowrey's change does fix the build. @Jeff-Lowrey would you be willing to propose a PR?

Jeff-Lowrey commented 4 years ago

@LucasRoesler Even though it's a very very quick fix, I probably don't have time in the next few days to work with a real PR.

LucasRoesler commented 4 years ago

Ok, i am will submit an update that will

1) standardize to waitress so that we don't have to worry about this, pure python all the way, and 2) update of-watchdog 3) and make sure each docker python image is on the latest 3.7, armf 3.6, and 2.7

martindekov commented 4 years ago

Closed in https://github.com/openfaas-incubator/python-flask-template/pull/33