search

pytries / marisa-trie

Static memory-efficient Trie-like structures for Python based on marisa-trie C++ library.
https://marisa-trie.readthedocs.io/en/latest/
MIT License
1.04k stars 92 forks source link

Build failure under Python-3.9 #59

Closed daa closed 3 years ago

daa commented 4 years ago

I tried to build marisa-trie with python-3.9 and failed with following error:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -Imarisa-trie/include -I/opt/python3.9/include/python3.9 -c src/marisa_trie.cpp -o build/temp.linux-x86_64-3.9/src/marisa_trie.o
  src/marisa_trie.cpp: In function ‘int __Pyx_modinit_type_init_code()’:
  src/marisa_trie.cpp:17944:34: error: ‘PyTypeObject {aka struct _typeobject}’ has no member named ‘tp_print’
     __pyx_type_11marisa_trie__Trie.tp_print = 0;

From documentation follows that tp_print was removed in Python-3.9: https://docs.python.org/dev/whatsnew/3.9.html#id3 .

daa commented 3 years ago

To solve the issue I tried to update cpp files with newer Cython (0.29.21) and it had success - I could successfully build wheels for python3.9 and older pythons too.

ygorg commented 3 years ago

@daa That's great ! How would one process to update the cpp files ? Is there a command to launch ? Is it simple ?

daa commented 3 years ago

@ygorg update_cpp.sh in marisa-trie source tree, requires installed Cython.

jay-pee commented 3 years ago

have also a problem with python 3.9 but maybe it is also a problem of Mac OS Big Sur and the M1 processor?

Tried to update ... didn't work:

(venv)  jaypee ~/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master  ./update_cpp.sh marisa-trie

/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/marisa_trie.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/agent.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/base.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
warning: src/base.pxd:5:4: 'marisa_num_tries' redeclared
warning: src/base.pxd:14:4: 'marisa_cache_level' redeclared
warning: src/base.pxd:23:4: 'marisa_tail_mode' redeclared
warning: src/base.pxd:40:4: 'marisa_node_order' redeclared
warning: src/base.pxd:52:4: 'marisa_config_mask' redeclared
warning: src/base.pxd:6:8: 'MARISA_MIN_NUM_TRIES' redeclared
warning: src/base.pxd:7:8: 'MARISA_MAX_NUM_TRIES' redeclared
warning: src/base.pxd:8:8: 'MARISA_DEFAULT_NUM_TRIES' redeclared
warning: src/base.pxd:15:8: 'MARISA_HUGE_CACHE' redeclared
warning: src/base.pxd:16:8: 'MARISA_LARGE_CACHE' redeclared
warning: src/base.pxd:17:8: 'MARISA_NORMAL_CACHE' redeclared
warning: src/base.pxd:18:8: 'MARISA_SMALL_CACHE' redeclared
warning: src/base.pxd:19:8: 'MARISA_TINY_CACHE' redeclared
warning: src/base.pxd:20:8: 'MARISA_DEFAULT_CACHE' redeclared
warning: src/base.pxd:28:8: 'MARISA_TEXT_TAIL' redeclared
warning: src/base.pxd:34:8: 'MARISA_BINARY_TAIL' redeclared
warning: src/base.pxd:36:8: 'MARISA_DEFAULT_TAIL' redeclared
warning: src/base.pxd:44:8: 'MARISA_LABEL_ORDER' redeclared
warning: src/base.pxd:49:8: 'MARISA_WEIGHT_ORDER' redeclared
warning: src/base.pxd:50:8: 'MARISA_DEFAULT_ORDER' redeclared
warning: src/base.pxd:53:8: 'MARISA_NUM_TRIES_MASK' redeclared
warning: src/base.pxd:54:8: 'MARISA_CACHE_LEVEL_MASK' redeclared
warning: src/base.pxd:55:8: 'MARISA_TAIL_MODE_MASK' redeclared
warning: src/base.pxd:56:8: 'MARISA_NODE_ORDER_MASK' redeclared
warning: src/base.pxd:57:8: 'MARISA_CONFIG_MASK' redeclared
warning: src/base.pxd:61:4: 'CacheLevel' redeclared
warning: src/base.pxd:62:4: 'TailMode' redeclared
warning: src/base.pxd:63:4: 'NodeOrder' redeclared
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/iostream.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/key.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/keyset.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/query.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/std_iostream.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)
/Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master/src/trie.pxd
  tree = Parsing.p_module(s, pxd, full_module_name)

(venv)  ✘ jaypee ~/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master  pip install .

Processing /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/marisa-trie-master
Using legacy 'setup.py install' for marisa-trie, since package 'wheel' is not installed.
Installing collected packages: marisa-trie
    Running setup.py install for marisa-trie ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3n/v_6smg1x7rlgh190rvbhy82r0000gn/T/pip-req-build-dr8qztdf/setup.py'"'"'; __file__='"'"'/private/var/folders/3n/v_6smg1x7rlgh190rvbhy82r0000gn/T/pip-req-build-dr8qztdf/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3n/v_6smg1x7rlgh190rvbhy82r0000gn/T/pip-record-xwo0j5mr/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/include/site/python3.9/marisa-trie
         cwd: /private/var/folders/3n/v_6smg1x7rlgh190rvbhy82r0000gn/T/pip-req-build-dr8qztdf/
    Complete output (19 lines):
    WARNING: The wheel package is not available.
    running install
    running build
    running build_clib
    building 'libmarisa-trie' library
    creating build
    creating build/temp.macosx-11-x86_64-3.9
    ar rcs build/temp.macosx-11-x86_64-3.9/liblibmarisa-trie.a
    ar: no archive members specified
    usage:  ar -d [-TLsv] archive file ...
        ar -m [-TLsv] archive file ...
        ar -m [-abiTLsv] position archive file ...
        ar -p [-TLsv] archive [file ...]
        ar -q [-cTLsv] archive file ...
        ar -r [-cuTLsv] archive file ...
        ar -r [-abciuTLsv] position archive file ...
        ar -t [-TLsv] archive [file ...]
        ar -x [-ouTLsv] archive [file ...]
    error: command '/usr/bin/ar' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3n/v_6smg1x7rlgh190rvbhy82r0000gn/T/pip-req-build-dr8qztdf/setup.py'"'"'; __file__='"'"'/private/var/folders/3n/v_6smg1x7rlgh190rvbhy82r0000gn/T/pip-req-build-dr8qztdf/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3n/v_6smg1x7rlgh190rvbhy82r0000gn/T/pip-record-xwo0j5mr/install-record.txt --single-version-externally-managed --compile --install-headers /Users/jaypee/Downloads/makeAnkiNotesFromCsv-master/venv/include/site/python3.9/marisa-trie Check the logs for full command output.
rlaphoenix commented 3 years ago

For those on arch-linux:

# pacman -S --needed build-devel cython python-pip
# pip install wheel
$ cd /tmp
$ git clone --recurse-submodules https://github.com/pytries/marisa-trie
$ cd marisa-trie
$ ./update_cpp.sh
$ pip install .

This updates cython references and installs from source.

rushter commented 3 years ago

Since this project is unmaintained, I've published a fork that supports Python 3.9 and has prebuilt wheels:

https://pypi.org/project/marisa-trie-m/

AlanMaxxx commented 3 years ago

I meet the same questions, when I use pip install marisa-trie, it doesn't work on Python3.9 and Windows10. The error is below. issue59

But finally, I solve it! Downloading .whl file from marisa-trie.whl . Next, installing directly from pip install xxx.whl . Done ,and you will see Successfully installed marisa-trie-xxx as below, installed

BoboTiG commented 3 years ago

Thanks a lot @rushter 💪

BoboTiG commented 3 years ago

Have a look at #66 ;)

BoboTiG commented 3 years ago

Fixed in master (b8f0cfd49a53914eb6c92f8d6d26f7cf366f8042).