search

nlpsandbox / data-node

Repository of FHIR and annotation resources used to benchmark NLP Sandbox tools
https://nlpsandbox.io
Apache License 2.0
4 stars 2 forks source link

/libpython3.8.a’ generated with LTO version 6.0 instead of the expected 7.1 #94

Closed tschaffter closed 3 years ago

tschaffter commented 3 years ago

Running pip install -r requirements.txt fails when running in a conda environment.

plugins/router_hash/router_hash.o is up to date
    plugins/router_expires/expires.o is up to date
    plugins/router_metrics/plugin.o is up to date
    plugins/transformation_template/tt.o is up to date
    plugins/stats_pusher_socket/plugin.o is up to date
    *** uWSGI linking ***
    gcc -pthread -B /home/tschaffter/.conda/envs/tox/compiler_compat -Wl,--sysroot=/ -o /mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/.tox/py38/bin/uwsgi  core/utils.o core/protocol.o core/socket.o core/logging.o core/master.o core/master_utils.o core/emperor.o core/notify.o core/mule.o core/subscription.o core/stats.o core/sendfile.o core/async.o core/master_checks.o core/fifo.o core/offload.o core/io.o core/static.o core/websockets.o core/spooler.o core/snmp.o core/exceptions.o core/config.o core/setup_utils.o core/clock.o core/init.o core/buffer.o core/reader.o core/writer.o core/alarm.o core/cron.o core/hooks.o core/plugins.o core/lock.o core/cache.o core/daemons.o core/errors.o core/hash.o core/master_events.o core/chunked.o core/queue.o core/event.o core/signal.o core/strings.o core/progress.o core/timebomb.o core/ini.o core/fsmon.o core/mount.o core/metrics.o core/plugins_builder.o core/sharedarea.o core/rpc.o core/gateway.o core/loop.o core/cookie.o core/querystring.o core/rb_timers.o core/transformations.o core/uwsgi.o proto/base.o proto/uwsgi.o proto/http.o proto/fastcgi.o proto/scgi.o proto/puwsgi.o lib/linux_ns.o core/yaml.o core/dot_h.o core/config_py.o plugins/python/python_plugin.o plugins/python/pyutils.o plugins/python/pyloader.o plugins/python/wsgi_handlers.o plugins/python/wsgi_headers.o plugins/python/wsgi_subhandler.o plugins/python/web3_subhandler.o plugins/python/pump_subhandler.o plugins/python/gil.o plugins/python/uwsgi_pymodule.o plugins/python/profiler.o plugins/python/symimporter.o plugins/python/tracebacker.o plugins/python/raw.o plugins/gevent/gevent.o plugins/gevent/hooks.o plugins/ping/ping_plugin.o plugins/cache/cache.o plugins/nagios/nagios.o plugins/rrdtool/rrdtool.o plugins/carbon/carbon.o plugins/rpc/rpc_plugin.o plugins/corerouter/cr_common.o plugins/corerouter/cr_map.o plugins/corerouter/corerouter.o plugins/fastrouter/fastrouter.o plugins/http/http.o plugins/http/keepalive.o plugins/http/https.o plugins/http/spdy3.o plugins/ugreen/ugreen.o plugins/signal/signal_plugin.o plugins/syslog/syslog_plugin.o plugins/rsyslog/rsyslog_plugin.o plugins/logsocket/logsocket_plugin.o plugins/router_uwsgi/router_uwsgi.o plugins/router_redirect/router_redirect.o plugins/router_basicauth/router_basicauth.o plugins/zergpool/zergpool.o plugins/redislog/redislog_plugin.o plugins/mongodblog/mongodblog_plugin.o plugins/router_rewrite/router_rewrite.o plugins/router_http/router_http.o plugins/logfile/logfile.o plugins/router_cache/router_cache.o plugins/rawrouter/rawrouter.o plugins/router_static/router_static.o plugins/sslrouter/sslrouter.o plugins/spooler/spooler_plugin.o plugins/cheaper_busyness/cheaper_busyness.o plugins/symcall/symcall_plugin.o plugins/transformation_tofile/tofile.o plugins/transformation_gzip/gzip.o plugins/transformation_chunked/chunked.o plugins/transformation_offload/offload.o plugins/router_memcached/router_memcached.o plugins/router_redis/router_redis.o plugins/router_hash/router_hash.o plugins/router_expires/expires.o plugins/router_metrics/plugin.o plugins/transformation_template/tt.o plugins/stats_pusher_socket/plugin.o -lpthread -lm -rdynamic -ldl -lpthread -ldl -lutil -lrt -lm /home/tschaffter/.conda/envs/tox/lib/python3.8/config-3.8-x86_64-linux-gnu/libpython3.8.a -lutil -lcrypt
    lto1: fatal error: bytecode stream in file ‘/home/tschaffter/.conda/envs/tox/lib/python3.8/config-3.8-x86_64-linux-gnu/libpython3.8.a’ generated with LTO version 6.0 instead of the expected 7.1
    compilation terminated.
    lto-wrapper: fatal error: gcc returned 1 exit status
    compilation terminated.
    /home/tschaffter/.conda/envs/tox/compiler_compat/ld: error: lto-wrapper failed
    collect2: error: ld returned 1 exit status
    *** error linking uWSGI ***
    ----------------------------------------
ERROR: Command errored out with exit status 1: /mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/.tox/py38/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8rdj5l3c/uwsgi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8rdj5l3c/uwsgi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-h4nylqv0/install-record.txt --single-version-externally-managed --compile --install-headers /mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/.tox/py38/include/site/python3.8/uWSGI Check the logs for full command output.

=============================================================================== log end ===============================================================================
ERROR: could not install deps [-r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/requirements.txt, -r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/test-requirements.txt]; v = InvocationError('/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/.tox/py38/bin/python -m pip install -r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/requirements.txt -r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/test-requirements.txt', 1)
_______________________________________________________________________________ summary _______________________________________________________________________________
ERROR:   py38: could not install deps [-r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/requirements.txt, -r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/test-requirements.txt]; v = InvocationError('/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/.tox/py38/bin/python -m pip install -r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/requirements.txt -r/mnt/c/Users/thoma/Documents/dev/nlpsandbox/data-node/server/test-requirements.txt', 1)
(tox) tschaffter@DESKTOP-H29C5A0:~/dev/nlpsandbox/data-node/server$ gcc --version
gcc (Debian 8.3.0-6) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

When the CI workflow runs (without using conda), pip install -r requirements.txt works.

tschaffter commented 3 years ago

The failing command can be reduce to pip install uwsgi in this conda environment:

$ python --version
Python 3.8.5
$ conda list
# packages in environment at /home/tschaffter/.conda/envs/tox:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
appdirs                   1.4.4                    pypi_0    pypi
ca-certificates           2020.10.14                    0
certifi                   2020.11.8        py38h06a4308_0
chardet                   3.0.4                    pypi_0    pypi
connexion                 2.7.0                    pypi_0    pypi
distlib                   0.3.1                    pypi_0    pypi
filelock                  3.0.12                   pypi_0    pypi
idna                      2.10                     pypi_0    pypi
ld_impl_linux-64          2.33.1               h53a641e_7
libedit                   3.1.20191231         h14c3975_1
libffi                    3.3                  he6710b0_2
libgcc-ng                 9.1.0                hdf63c60_0
libstdcxx-ng              9.1.0                hdf63c60_0
mongoengine               0.21.0                   pypi_0    pypi
ncurses                   6.2                  he6710b0_1
openssl                   1.1.1h               h7b6447c_0
packaging                 20.7                     pypi_0    pypi
pip                       20.2.4           py38h06a4308_0
pluggy                    0.13.1                   pypi_0    pypi
psycopg2-binary           2.8.6                    pypi_0    pypi
py                        1.9.0                    pypi_0    pypi
pymongo                   3.11.1                   pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
python                    3.8.5                h7579374_1
readline                  8.0                  h7b6447c_0
requests                  2.25.0                   pypi_0    pypi
setuptools                50.3.1           py38h06a4308_1
sqlite                    3.33.0               h62c20be_0
swagger-ui-bundle         0.0.8                    pypi_0    pypi
tk                        8.6.10               hbc83047_0
toml                      0.10.2                   pypi_0    pypi
tox                       3.20.1                   pypi_0    pypi
urllib3                   1.26.2                   pypi_0    pypi
virtualenv                20.2.1                   pypi_0    pypi
wheel                     0.35.1             pyhd3eb1b0_0
xz                        5.2.5                h7b6447c_0
zlib                      1.2.11               h7b6447c_3
tschaffter commented 3 years ago

A solution seems to install a lower version of gcc, which I don't want to do: https://www.programmersought.com/article/24454048850/

In the past I've used the following command to install uwsgi using conda: conda install -c conda-forge uwsgi

After running the above command, pip install uwsgi works:

$ pip install uwsgi
Requirement already satisfied: uwsgi in /home/tschaffter/.conda/envs/tox/lib/python3.8/site-packages (2.0.19.1)

However ideally I don't want to have to install uwsgi using conda because the CI/CD workflow does not use conda. The CI workflow currently fails to run tox because tox create a virtualenvironment for py38 (see tox.ini) where uwsgi can not be installed.

tschaffter commented 3 years ago

I decided to remove uwsgi from requirements.txt and instead install it as part of the Dockerfile. Leaving uwsgi, used only for production deployment, would lead to too much confusion to the developer using conda, which we promote. If running uwsgi locally is required when using conda, it should be installed with conda install -c conda-forge uwsgi and not using pip install uwsgi.