search

opensemanticsearch / open-semantic-search

Open Source research tool to search, browse, analyze and explore large document collections by Semantic Search Engine and Open Source Text Mining & Text Analytics platform (Integrates ETL for document processing, OCR for images & PDF, named entity recognition for persons, organizations & locations, metadata management by thesaurus & ontologies, search user interface & search apps for fulltext search, faceted search & knowledge graph)
https://opensemanticsearch.org
GNU General Public License v3.0
974 stars 169 forks source link

Docker-compose build: Error when building multidict #318

Closed certrik closed 4 years ago

certrik commented 4 years ago

When running docker-compose build it fails at buildling multidict with the errors below ...

Running setup.py bdist_wheel for multidict: started
  Running setup.py bdist_wheel for multidict: finished with status 'error'
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-oeubtpbp/multidict/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-i7_5yrko --python-tag cp37:
  **********************
  * Accellerated build *
  **********************
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/multidict
  copying multidict/_abc.py -> build/lib.linux-x86_64-3.7/multidict
  copying multidict/_multidict_base.py -> build/lib.linux-x86_64-3.7/multidict
  copying multidict/_compat.py -> build/lib.linux-x86_64-3.7/multidict
  copying multidict/__init__.py -> build/lib.linux-x86_64-3.7/multidict
  copying multidict/_multidict_py.py -> build/lib.linux-x86_64-3.7/multidict
  running egg_info
  writing multidict.egg-info/PKG-INFO
  writing dependency_links to multidict.egg-info/dependency_links.txt
  writing top-level names to multidict.egg-info/top_level.txt
  reading manifest file 'multidict.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no previously-included files matching '*.pyc' found anywhere in distribution
  warning: no previously-included files found matching 'multidict/_multidict.html'
  warning: no previously-included files found matching 'multidict/*.so'
  warning: no previously-included files found matching 'multidict/*.pyd'
  warning: no previously-included files found matching 'multidict/*.pyd'
  no previously-included directories found matching 'docs/_build'
  writing manifest file 'multidict.egg-info/SOURCES.txt'
  copying multidict/__init__.pyi -> build/lib.linux-x86_64-3.7/multidict
  copying multidict/_multidict.c -> build/lib.linux-x86_64-3.7/multidict
  copying multidict/py.typed -> build/lib.linux-x86_64-3.7/multidict
  creating build/lib.linux-x86_64-3.7/multidict/_multilib
  copying multidict/_multilib/defs.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
  copying multidict/_multilib/dict.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
  copying multidict/_multilib/istr.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
  copying multidict/_multilib/iter.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
  copying multidict/_multilib/pair_list.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
  copying multidict/_multilib/views.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
  running build_ext
  building 'multidict._multidict' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/multidict
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.7m -c multidict/_multidict.c -o build/temp.linux-x86_64-3.7/multidict/_multidict.o -O2 -std=c99 -Wall -Wsign-compare -Wconversion -fno-strict-aliasing -pedantic
  unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for multidict
  Running setup.py clean for multidict
  Running setup.py bdist_wheel for yarl: started
  Running setup.py bdist_wheel for yarl: finished with status 'error'
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-oeubtpbp/yarl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-utva4jpo --python-tag cp37:
  **********************
  * Accellerated build *
  **********************
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/yarl
  copying yarl/_quoting.py -> build/lib.linux-x86_64-3.7/yarl
  copying yarl/_quoting_py.py -> build/lib.linux-x86_64-3.7/yarl
  copying yarl/_url.py -> build/lib.linux-x86_64-3.7/yarl
  copying yarl/__init__.py -> build/lib.linux-x86_64-3.7/yarl
  running egg_info
  writing yarl.egg-info/PKG-INFO
  writing dependency_links to yarl.egg-info/dependency_links.txt
  writing requirements to yarl.egg-info/requires.txt
  writing top-level names to yarl.egg-info/top_level.txt
  reading manifest file 'yarl.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no previously-included files matching '*.pyc' found anywhere in distribution
  warning: no previously-included files matching '*.cache' found anywhere in distribution
  warning: no previously-included files found matching 'yarl/*.html'
  warning: no previously-included files found matching 'yarl/*.so'
  warning: no previously-included files found matching 'yarl/*.pyd'
  no previously-included directories found matching 'docs/_build'
  writing manifest file 'yarl.egg-info/SOURCES.txt'
  copying yarl/__init__.pyi -> build/lib.linux-x86_64-3.7/yarl
  copying yarl/_quoting_c.c -> build/lib.linux-x86_64-3.7/yarl
  copying yarl/_quoting_c.pyi -> build/lib.linux-x86_64-3.7/yarl
  copying yarl/_quoting_c.pyx -> build/lib.linux-x86_64-3.7/yarl
  copying yarl/py.typed -> build/lib.linux-x86_64-3.7/yarl
  running build_ext
  building 'yarl._quoting_c' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/yarl
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.7m -c yarl/_quoting_c.c -o build/temp.linux-x86_64-3.7/yarl/_quoting_c.o
  unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for yarl
  Running setup.py clean for yarl
Successfully built py2neo tika twint PyDispatcher protego aiohttp fake-useragent googletransx
Failed to build multidict yarl
Installing collected packages: py2neo, constantly, hyperlink, zope.interface, PyHamcrest, attrs, Automat, incremental, Twisted, pyasn1, pyasn1-modules, pycparser, cffi, cryptography, service-identity, pyOpenSSL, w3lib, PyDispatcher, queuelib, cssselect, parsel, itemadapter, jmespath, itemloaders, protego, scrapy, warcio, tika, SPARQLWrapper, pycares, aiodns, multidict, async-timeout, typing-extensions, yarl, aiohttp, python-socks, aiohttp-socks, soupsieve, beautifulsoup4, cchardet, elasticsearch, fake-useragent, geographiclib, geopy, googletransx, numpy, pandas, pysocks, schedule, twint
  Running setup.py install for multidict: started
    Running setup.py install for multidict: finished with status 'error'
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-oeubtpbp/multidict/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-534g68vj/install-record.txt --single-version-externally-managed --compile:
    **********************
    * Accellerated build *
    **********************
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/multidict
    copying multidict/_abc.py -> build/lib.linux-x86_64-3.7/multidict
    copying multidict/_multidict_base.py -> build/lib.linux-x86_64-3.7/multidict
    copying multidict/_compat.py -> build/lib.linux-x86_64-3.7/multidict
    copying multidict/__init__.py -> build/lib.linux-x86_64-3.7/multidict
    copying multidict/_multidict_py.py -> build/lib.linux-x86_64-3.7/multidict
    running egg_info
    writing multidict.egg-info/PKG-INFO
    writing dependency_links to multidict.egg-info/dependency_links.txt
    writing top-level names to multidict.egg-info/top_level.txt
    reading manifest file 'multidict.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files found matching 'multidict/_multidict.html'
    warning: no previously-included files found matching 'multidict/*.so'
    warning: no previously-included files found matching 'multidict/*.pyd'
    warning: no previously-included files found matching 'multidict/*.pyd'
    no previously-included directories found matching 'docs/_build'
    writing manifest file 'multidict.egg-info/SOURCES.txt'
    copying multidict/__init__.pyi -> build/lib.linux-x86_64-3.7/multidict
    copying multidict/_multidict.c -> build/lib.linux-x86_64-3.7/multidict
    copying multidict/py.typed -> build/lib.linux-x86_64-3.7/multidict
    creating build/lib.linux-x86_64-3.7/multidict/_multilib
    copying multidict/_multilib/defs.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
    copying multidict/_multilib/dict.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
    copying multidict/_multilib/istr.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
    copying multidict/_multilib/iter.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
    copying multidict/_multilib/pair_list.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
    copying multidict/_multilib/views.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
    running build_ext
    building 'multidict._multidict' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/multidict
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.7m -c multidict/_multidict.c -o build/temp.linux-x86_64-3.7/multidict/_multidict.o -O2 -std=c99 -Wall -Wsign-compare -Wconversion -fno-strict-aliasing -pedantic
    unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-oeubtpbp/multidict/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-534g68vj/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-oeubtpbp/multidict/
ERROR: Service 'etl' failed to build: The command '/bin/sh -c pip3 install -r /usr/lib/python3/dist-packages/opensemanticetl/requirements.txt' returned a non-zero code: 1
Mandalka commented 4 years ago

This dependency issue on newest Debian/Ubuntu releases was fixed some days ago by adding the new dependencies to Docker container. I upgraded the git submodule reference in this repo to this new Open Semantic ETL commit/release.

In some days there will be some more CI/CD pipes for more / different and coming Linux environments, so such things should happen less often (and until then maybe yet another dependency hell issues, which will be fixed next days by setting up and debugging more fully automatic tests for additional/new Linux environments).