search

infoscout / weighted-levenshtein

Weighted Levenshtein library
MIT License
105 stars 26 forks source link

Installation via PyPi does not work with Python 3.9 #25

Closed MartinNiederl closed 1 year ago

MartinNiederl commented 3 years ago

When trying to install via pip install weighted-levenshtein using Python 3.9 the following error occurs multiple times:

weighted_levenshtein\clev.c(20188): error C2039: "tp_print": is not a member of "_typeobject"
D:\...\include\cpython/object.h(193): note: see declaration of '_typeobject'

As I read in this StackOverflow answer, tp_print was removed in Python 3.9. (some more details: here)

I am absolutely not familiar with Cython because I have never used it before, however I was able to fix the problem by cloning the project, installing a more recent version of Cython (0.29.24) and simply installed weighted-levenshtein locally with pip install ..

So maybe it is enough to increase the version in requirements.txt.

pachewise commented 3 years ago

@MartinNiederl thanks for bringing this up. Right now we only support up to 3.6, but if you can you open a PR for the changes required to get the tests passing on 3.9, we can review to merge and add 3.9 to the CI checks.

woctezuma commented 3 years ago

Using Python 3.9.6, I encounter the same error as Martin when running:

pip install weighted-levenshtein

However, without any need to modify anything, I can install the package, then run all the tests perfectly fine with:

git clone https://github.com/infoscout/weighted-levenshtein.git
cd weighted-levenshtein
pip install .

followed by:

pip install -r requirements.txt
pip install coveralls
coverage run setup.py test

which returns:

running test
[...]
copying build\lib.win-amd64-3.9\weighted_levenshtein\clev.cp39-win_amd64.pyd -> weighted_levenshtein
test_dl (test.test.TestClev) ... ok
test_dl_transpose (test.test.TestClev) ... ok
test_dl_transpose2 (test.test.TestClev) ... ok
test_dl_transpose3 (test.test.TestClev) ... ok
test_dl_transpose4 (test.test.TestClev) ... ok
test_lev (test.test.TestClev) ... ok
test_lev_delete (test.test.TestClev) ... ok
test_lev_insert (test.test.TestClev) ... ok
test_lev_substitute (test.test.TestClev) ... ok
test_osa (test.test.TestClev) ... ok
test_osa_delete (test.test.TestClev) ... ok
test_osa_insert (test.test.TestClev) ... ok
test_osa_substitute (test.test.TestClev) ... ok
test_osa_transpose (test.test.TestClev) ... ok
test_dl (test.test.TestClevUsingDefaultValues) ... ok
test_lev (test.test.TestClevUsingDefaultValues) ... ok
test_osa (test.test.TestClevUsingDefaultValues) ... ok

----------------------------------------------------------------------
Ran 17 tests in 0.012s

OK

I think you should be able to support version 3.9 on PyPI without any change, other than building the wheels with CircleCI.

rbagd commented 2 years ago

Hi, is there any news on this? As @woctezuma, I didn't have any issues using weighted-levenshtein by cloning the repo and doing a pip install on recent Python versions. It'd be nicer though to have it straight from PyPi, as the work-around overcomplicates Dockerfile setup.

woctezuma commented 2 years ago

For reference, you might want to have a look at https://github.com/maxbachmann/RapidFuzz

Maybe this library does not support weights, I don't know.

maxbachmann commented 2 years ago

For reference, you might want to have a look at https://github.com/maxbachmann/RapidFuzz

rapidfuzz does not support OSA and Damerau Levenshtein yet, but support is in works.

Maybe this library does not support weights, I don't know.

Right now I only support fixed weights (e.g. giving all substitutions a weight of 2), but character dependent weights are planned: https://github.com/maxbachmann/RapidFuzz/issues/241

jerzyorlowskimim commented 2 years ago

I try to install weighted-weighted-levenshtein on python 3.9 and get. errors below.

error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1


  │ exit code: 1
  ╰─> [145 lines of output]
      /home/jerzy/PycharmProjects/MiM/invicta_data_platform/venv2/lib/python3.9/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      running install
      /home/jerzy/PycharmProjects/MiM/invicta_data_platform/venv2/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      warning: build_py: byte-compiling is disabled, skipping.

      running build_ext
      skipping 'weighted_levenshtein/clev.c' Cython extension (up-to-date)
      building 'weighted_levenshtein.clev' extension
      weighted_levenshtein/clev.c: In function ‘__Pyx_modinit_type_init_code’:
      weighted_levenshtein/clev.c:20188:25: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
      20188 |   __pyx_type___pyx_array.tp_print = 0;
            |                         ^
      weighted_levenshtein/clev.c:20193:31: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
      20193 |   __pyx_type___pyx_MemviewEnum.tp_print = 0;
            |                               ^
      weighted_levenshtein/clev.c:20208:30: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
      20208 |   __pyx_type___pyx_memoryview.tp_print = 0;
            |                              ^
      weighted_levenshtein/clev.c:20221:35: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
      20221 |   __pyx_type___pyx_memoryviewslice.tp_print = 0;
            |                                   ^
      weighted_levenshtein/clev.c: In function ‘__Pyx_ParseOptionalKeywords’:
      weighted_levenshtein/clev.c:21303:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21303 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21303:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      21303 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21303:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21303 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21303:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21303 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21303:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      21303 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21303:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21303 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21319:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21319 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21319:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      21319 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21319:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21319 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21319:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21319 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21319:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      21319 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c:21319:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      21319 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      weighted_levenshtein/clev.c: In function ‘__Pyx_decode_c_string’:
      weighted_levenshtein/clev.c:22209:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
      22209 |         return PyUnicode_FromUnicode(NULL, 0);
            |         ^~~~~~
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from weighted_levenshtein/clev.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:551:42: note: declared here
        551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
            |                                          ^~~~~~~~~~~~~~~~~~~~~
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> weighted-levenshtein

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.```
JacobHayes commented 2 years ago

I think the reason this fails when installing from pypi, but not when installing from the repo, is because the sdist in pypi includes the generated .c file, which is somewhat python version specific. The Cython docs seem to suggest different things at different points, but with limited releases, it might be best to require Cython on installation and omit the .c files in the distributions. I can make a PR for that in case it gets merged.

JacobHayes commented 1 year ago

Thanks for merging #31 @JimReed!

Do you think you'd be able to cut a new release and upload to pypi? At the least, something like this should handle the pypi side:

pip3 install twine
python3 setup.py sdist
twine upload dist/weighted_levenshtein-0.2.2.tar.gz # Will need PyPI credentials valid for the package
KrisKennaway commented 1 year ago

Hi, I'm also interested in seeing a new PyPi release that works on 3.9+ - this dependency is the only blocker for updating one of my projects. Is this in the works?