search

facebookresearch / fastText

Library for fast text representation and classification.
https://fasttext.cc/
MIT License
25.83k stars 4.71k forks source link

Build fails with Python 3.10 #1298

Open thaneesh-babu opened 1 year ago

thaneesh-babu commented 1 year ago

Problem

Tried installing fastText with Python 3.10 and it results in a build failure.

Input

pip install fasttext

Output

Defaulting to user installation because normal site-packages is not writeable
Collecting fasttext
  Using cached fasttext-0.9.2.tar.gz (68 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: pybind11>=2.2 in ./.local/lib/python3.10/site-packages (from fasttext) (2.10.0)
Requirement already satisfied: setuptools>=0.7.0 in /usr/lib/python3.10/site-packages (from fasttext) (59.6.0)
Requirement already satisfied: numpy in /usr/lib64/python3.10/site-packages (from fasttext) (1.22.0)
Building wheels for collected packages: fasttext
  Building wheel for fasttext (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-vxzd8whp
       cwd: /tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/
  Complete output (39 lines):
  /usr/lib/python3.10/site-packages/setuptools/dist.py:723: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
    warnings.warn(
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.10
  creating build/lib.linux-x86_64-3.10/fasttext
  copying python/fasttext_module/fasttext/__init__.py -> build/lib.linux-x86_64-3.10/fasttext
  copying python/fasttext_module/fasttext/FastText.py -> build/lib.linux-x86_64-3.10/fasttext
  creating build/lib.linux-x86_64-3.10/fasttext/util
  copying python/fasttext_module/fasttext/util/util.py -> build/lib.linux-x86_64-3.10/fasttext/util
  copying python/fasttext_module/fasttext/util/__init__.py -> build/lib.linux-x86_64-3.10/fasttext/util
  creating build/lib.linux-x86_64-3.10/fasttext/tests
  copying python/fasttext_module/fasttext/tests/test_script.py -> build/lib.linux-x86_64-3.10/fasttext/tests
  copying python/fasttext_module/fasttext/tests/test_configurations.py -> build/lib.linux-x86_64-3.10/fasttext/tests
  copying python/fasttext_module/fasttext/tests/__init__.py -> build/lib.linux-x86_64-3.10/fasttext/tests
  running build_ext
  creating tmp
  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c /tmp/tmpxrhxx9vy.cpp -o tmp/tmpxrhxx9vy.o -std=c++14
  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c /tmp/tmpv9s_a9e2.cpp -o tmp/tmpv9s_a9e2.o -fvisibility=hidden
  building 'fasttext_pybind' extension
  creating build/temp.linux-x86_64-3.10
  creating build/temp.linux-x86_64-3.10/python
  creating build/temp.linux-x86_64-3.10/python/fasttext_module
  creating build/temp.linux-x86_64-3.10/python/fasttext_module/fasttext
  creating build/temp.linux-x86_64-3.10/python/fasttext_module/fasttext/pybind
  creating build/temp.linux-x86_64-3.10/src
  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/home/thans/.local/lib/python3.10/site-packages/pybind11/include -I/home/thans/.local/lib/python3.10/site-packages/pybind11/include -Isrc -I/usr/include/python3.10 -c python/fasttext_module/fasttext/pybind/fasttext_pybind.cc -o build/temp.linux-x86_64-3.10/python/fasttext_module/fasttext/pybind/fasttext_pybind.o -DVERSION_INFO="0.9.2" -std=c++14 -fvisibility=hidden
  In file included from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../attr.h:13,
                   from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/class.h:12,
                   from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/pybind11.h:13,
                   from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/numpy.h:12,
                   from python/fasttext_module/fasttext/pybind/fasttext_pybind.cc:13:
  /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../detail/common.h:212:10: fatal error: Python.h: No such file or directory
    212 | #include <Python.h>
        |          ^~~~~~~~~~
  compilation terminated.
  error: command '/usr/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for fasttext
  Running setup.py clean for fasttext
Failed to build fasttext
Installing collected packages: fasttext
    Running setup.py install for fasttext ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-gubaknot/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/thans/.local/include/python3.10/fasttext
         cwd: /tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/
    Complete output (40 lines):
    /usr/lib/python3.10/site-packages/setuptools/dist.py:723: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
      warnings.warn(
    running install
    /usr/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.10
    creating build/lib.linux-x86_64-3.10/fasttext
    copying python/fasttext_module/fasttext/__init__.py -> build/lib.linux-x86_64-3.10/fasttext
    copying python/fasttext_module/fasttext/FastText.py -> build/lib.linux-x86_64-3.10/fasttext
    creating build/lib.linux-x86_64-3.10/fasttext/util
    copying python/fasttext_module/fasttext/util/util.py -> build/lib.linux-x86_64-3.10/fasttext/util
    copying python/fasttext_module/fasttext/util/__init__.py -> build/lib.linux-x86_64-3.10/fasttext/util
    creating build/lib.linux-x86_64-3.10/fasttext/tests
    copying python/fasttext_module/fasttext/tests/test_script.py -> build/lib.linux-x86_64-3.10/fasttext/tests
    copying python/fasttext_module/fasttext/tests/test_configurations.py -> build/lib.linux-x86_64-3.10/fasttext/tests
    copying python/fasttext_module/fasttext/tests/__init__.py -> build/lib.linux-x86_64-3.10/fasttext/tests
    running build_ext
    gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c /tmp/tmpx6gd29uw.cpp -o tmp/tmpx6gd29uw.o -std=c++14
    gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.10 -c /tmp/tmpf9b7kvnc.cpp -o tmp/tmpf9b7kvnc.o -fvisibility=hidden
    building 'fasttext_pybind' extension
    creating build/temp.linux-x86_64-3.10
    creating build/temp.linux-x86_64-3.10/python
    creating build/temp.linux-x86_64-3.10/python/fasttext_module
    creating build/temp.linux-x86_64-3.10/python/fasttext_module/fasttext
    creating build/temp.linux-x86_64-3.10/python/fasttext_module/fasttext/pybind
    creating build/temp.linux-x86_64-3.10/src
    gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/home/thans/.local/lib/python3.10/site-packages/pybind11/include -I/home/thans/.local/lib/python3.10/site-packages/pybind11/include -Isrc -I/usr/include/python3.10 -c python/fasttext_module/fasttext/pybind/fasttext_pybind.cc -o build/temp.linux-x86_64-3.10/python/fasttext_module/fasttext/pybind/fasttext_pybind.o -DVERSION_INFO="0.9.2" -std=c++14 -fvisibility=hidden
    In file included from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../attr.h:13,
                     from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/class.h:12,
                     from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/pybind11.h:13,
                     from /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/numpy.h:12,
                     from python/fasttext_module/fasttext/pybind/fasttext_pybind.cc:13:
    /home/thans/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../detail/common.h:212:10: fatal error: Python.h: No such file or directory
      212 | #include <Python.h>
          |          ^~~~~~~~~~
    compilation terminated.
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nwoorord/fasttext_45c07b929fc14f92934445a99979270e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-gubaknot/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/thans/.local/include/python3.10/fasttext Check the logs for full command output.

Other Issues

I did go over #1081 to resolve my problem and I have the following requirements installed already: gcc gcc-c++ kernel-devel python-devel libxslt-devel libffi-devel openssl-devel I still get the error however.

System

I'm currently running Fedora 36, 5.17.5-300.fc36.x86_64

qathom commented 1 year ago

Hi @thaneesh-babu I just faced this issue earlier today :-) I could fix it by installing fasttext-wheel instead of fasttext. Try: poetry add fasttext-wheel

FinnWoelm commented 1 year ago

If you are encountering this issue with poetry and fasttext, check out #1292 for context and workarounds.