search

githubharald / CTCWordBeamSearch

Connectionist Temporal Classification (CTC) decoder with dictionary and language model.
https://towardsdatascience.com/b051d28f3d2e
MIT License
557 stars 160 forks source link

Python version #72

Closed tclewis29 closed 8 months ago

tclewis29 commented 1 year ago

What is the most recent version of Python that is compatible with CTCWordBeamSearch? I am using Python 3.11.1 and I am getting this error, which I believe is a compatibility issue.

ltd\documents\research\datasheet project\new_env\lib\site-packages (from word-beam-search==1.0.1) (1.26.1) Building wheels for collected packages: word-beam-search Building wheel for word-beam-search (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [36 lines of output] running bdist_wheel running build running build_ext building 'word_beam_search' extension creating build creating build\temp.win-amd64-cpython-311 creating build\temp.win-amd64-cpython-311\Release creating build\temp.win-amd64-cpython-311\Release\cpp "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ "-ILtd\Documents\Research\Datasheet project\new_env\include" -IC:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include -IC:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" /EHsc /Tpcpp/Beam.cpp /Fobuild\temp.win-amd64-cpython-311\Release\cpp/Beam.obj Beam.cpp "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ "-IC:\Users\thomas.lewis\Documents\Research\Datasheet project\new_env\include" -IC:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include -IC:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" /EHsc /Tpcpp/LanguageModel.cpp /Fobuild\temp.win-amd64-cpython-311\Release\cpp/LanguageModel.obj LanguageModel.cpp "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ "-IC:\Documents\Research\Datasheet project\new_env\include" -IC:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include -IC:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" /EHsc /Tpcpp/NPWordBeamSearch.cpp /Fobuild\temp.win-amd64-cpython-311\Release\cpp/NPWordBeamSearch.obj NPWordBeamSearch.cpp C:\Users\thomas.lewis\OneDrive - Longleat Enterprises Ltd\Documents\Research\Datasheet project\CTCWordBeamSearch\cpp\pybind\pybind11\cast.h(446): error C2027: use of undefined type '_frame' C:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include\pytypedefs.h(22): note: see declaration of '_frame' C:\Users\thomas.lewis\OneDrive - Longleat Enterprises Ltd\Documents\Research\Datasheet project\CTCWordBeamSearch\cpp\pybind\pybind11\cast.h(448): error C2027: use of undefined type '_frame' C:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include\pytypedefs.h(22): note: see declaration of '_frame' C:\Users\thomas.lewis\OneDrive - Longleat Enterprises Ltd\Documents\Research\Datasheet project\CTCWordBeamSearch\cpp\pybind\pybind11\cast.h(449): error C2027: use of undefined type '_frame' C:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include\pytypedefs.h(22): note: see declaration of '_frame' cpp/pybind/pybind11/pybind11.h(2096): error C2039: 'frame': is not a member of '_ts' C:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include\cpython/pystate.h(82): note: see declaration of '_ts' cpp/pybind/pybind11/pybind11.h(2097): error C2027: use of undefined type '_frame' C:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include\pytypedefs.h(22): note: see declaration of '_frame' cpp/pybind/pybind11/pybind11.h(2098): error C2027: use of undefined type '_frame' cpp/pybind/pybind11/pybind11.h(2101): error C2027: use of undefined type '_frame' C:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include\pytypedefs.h(22): note: see declaration of '_frame' cpp/pybind/pybind11/pybind11.h(2100): error C2660: 'PyDict_GetItem': function does not take 1 arguments C:\Users\thomas.lewis\AppData\Local\Programs\Python\Python311\include\dictobject.h(22): note: see declaration of 'PyDict_GetItem' cpp/pybind/pybind11/pybind11.h(2100): note: while trying to match the argument list '()' cpp/pybind/pybind11/numpy.h(36): error C2065: 'ssize_t': undeclared identifier cpp/pybind/pybind11/numpy.h(36): error C2338: static_assert failed: 'ssize_t != Py_intptr_t' error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe' failed with exit code 2 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for word-beam-search Running setup.py clean for word-beam-search Failed to build word-beam-search

nunomrm commented 1 year ago

I got a very similar error (below). Did you find a solution?

Building wheels for collected packages: word-beam-search
  Building wheel for word-beam-search (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'word_beam_search' extension
      creating build
      creating build\temp.win-amd64-cpython-310
      creating build\temp.win-amd64-cpython-310\Release
      creating build\temp.win-amd64-cpython-310\Release\cpp
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ -IC:\ProgramData\anaconda3\include -IC:\ProgramData\anaconda3\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" /EHsc /Tpcpp/Beam.cpp /Fobuild\temp.win-amd64-cpython-310\Release\cpp/Beam.obj
      Beam.cpp
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ -IC:\ProgramData\anaconda3\include -IC:\ProgramData\anaconda3\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" /EHsc /Tpcpp/LanguageModel.cpp /Fobuild\temp.win-amd64-cpython-310\Release\cpp/LanguageModel.obj
      LanguageModel.cpp
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ -IC:\ProgramData\anaconda3\include -IC:\ProgramData\anaconda3\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" /EHsc /Tpcpp/NPWordBeamSearch.cpp /Fobuild\temp.win-amd64-cpython-310\Release\cpp/NPWordBeamSearch.obj
      NPWordBeamSearch.cpp
      cpp/pybind/pybind11/numpy.h(36): error C2065: 'ssize_t': undeclared identifier
      cpp/pybind/pybind11/numpy.h(36): error C2338: static_assert failed: 'ssize_t != Py_intptr_t'
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.35.32215\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for word-beam-search
  Running setup.py clean for word-beam-search
Failed to build word-beam-search
Installing collected packages: word-beam-search
  Running setup.py install for word-beam-search ... error
  error: subprocess-exited-with-error

  × Running setup.py install for word-beam-search did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      running install
      C:\ProgramData\anaconda3\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_ext
      building 'word_beam_search' extension
      creating build
      creating build\temp.win-amd64-cpython-310
      creating build\temp.win-amd64-cpython-310\Release
      creating build\temp.win-amd64-cpython-310\Release\cpp
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ -IC:\ProgramData\anaconda3\include -IC:\ProgramData\anaconda3\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" /EHsc /Tpcpp/Beam.cpp /Fobuild\temp.win-amd64-cpython-310\Release\cpp/Beam.obj
      Beam.cpp
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ -IC:\ProgramData\anaconda3\include -IC:\ProgramData\anaconda3\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" /EHsc /Tpcpp/LanguageModel.cpp /Fobuild\temp.win-amd64-cpython-310\Release\cpp/LanguageModel.obj
      LanguageModel.cpp
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icpp/pybind/ -IC:\ProgramData\anaconda3\include -IC:\ProgramData\anaconda3\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.7.2\include\um" /EHsc /Tpcpp/NPWordBeamSearch.cpp /Fobuild\temp.win-amd64-cpython-310\Release\cpp/NPWordBeamSearch.obj
      NPWordBeamSearch.cpp
      cpp/pybind/pybind11/numpy.h(36): error C2065: 'ssize_t': undeclared identifier
      cpp/pybind/pybind11/numpy.h(36): error C2338: static_assert failed: 'ssize_t != Py_intptr_t'
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.35.32215\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> word-beam-search

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
githubharald commented 12 months ago

could you try this PR: https://github.com/githubharald/CTCWordBeamSearch/pull/69 it seems that only the type "ssize_t" is unknown to the compiler on Windows.

As soon as I have time I will update all my repos to the newest Python version.

alienver1 commented 9 months ago

Downgrading Python from 3.11.5 to 3.9.18 worked for me.

jrbastien commented 9 months ago

The pull request #69 is only correcting the second issue reported by @nunomrm. It is not fixing the original issue that was raised here by @tclewis29. The problem is that the pybind11 version that is distributed with this code does not work with Python 3.11. I have done a pip install to get version 2.11.1 and copied the newly acquired libraries to ...\CTCWordBeamSearch-master\cpp\pybind\pybind11.

This worked for me.

githubharald commented 9 months ago

@jrbastien thanks for checking, so updating pybind does the trick? That's also the option I prefer, instead of changing the original pybind sources.

jrbastien commented 9 months ago

Using the sources provided in pybind 2.11.1 works with Python 3.11. Not sure now, that it will still work with older versions of Python. I haven't tested it.

githubharald commented 9 months ago

all clear, thanks.

githubharald commented 8 months ago

I updated to the newest pybind version, and successfully tested the decoder with python 3.11 and 3.12.

jrbastien commented 8 months ago

Thank you for this.