Fix installation issue and tests

[r9y9]

_warp_ctc module wasn't correctly installed by python setup.py install. Use setuptools to collect modules to be installed.

fixes espnet/espnet#115 (comment).

Also fixed tests.

In [1]: from setuptools import find_packages

In [2]: find_packages()
Out[2]: ['warpctc_pytorch', 'warpctc_pytorch._warp_ctc']

Fixes #30

[ryanleary]

LGTM

[r9y9]

@SeanNaren Could you take a look?

[SeanNaren]

Thanks man!

[han-sha]

Hi, I am still having problems importing warpctc_pytorch import CTCLoss says:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/Documents/warp-ctc/pytorch_binding/warpctc_pytorch/__init__.py", line 6, in <module>
    from ._warp_ctc import *
  File "/Users/user/Documents/warp-ctc/pytorch_binding/warpctc_pytorch/_warp_ctc/__init__.py", line 3, in <module>
    from .__warp_ctc import lib as _lib, ffi as _ffi
ModuleNotFoundError: No module named 'warpctc_pytorch._warp_ctc.__warp_ctc'

find_packages() showed me the correct output:

>>> from setuptools import find_packages
>>> find_packages()
['warpctc_pytorch', 'warpctc_pytorch._warp_ctc']

If I move to: cd /anaconda3/lib/python3.6/site-packages and do

python3
import torch
from warpctc_pytorch import CTCLoss

it works fine.

But everywhere else raises the importing error.

Thanks for your help!

[sailordiary]

@ehcecsei Try adding /anaconda3/lib/python3.6/site-packages to your system's PATH variable and see if it fixes the problem. Works for me.

[jdhao]

For me, I met the following error:

Traceback (most recent call last): File "", line 1, in File "/home/haojiedong/projects/warp-ctc/pytorch_binding/warpctc_pytorch/init.py", line 6, in from ._warp_ctc import * File "/home/haojiedong/projects/warp-ctc/pytorch_binding/warpctc_pytorch/_warp_ctc/init.py", line 3, in from .warp_ctc import lib as _lib, ffi as _ffi ModuleNotFoundError: No module named 'warpctc_pytorch._warp_ctc.warp_ctc'

The reason is that I forget to set WARP_CTC_PATH. After building the module successfully, add the following to your .bash_profile:

export WARP_CTC_PATH=/path_to_warpctc/warp_ctc/build

Source your .bash_profile to make the change take effect. Now, install the python package according to instruction and you should be fine.

[thierrydesot1]

hi, Whatever I try, taking into account the above remarks, I keep getting

Traceback (most recent call last): File "test_gpu.py", line 2, in import warpctc_pytorch as warp_ctc File "/home/getalp/desotth/virtenv_pytorch/anaconda3/lib/python3.6/site-packages/warpctc_pytorch/init.py", line 6, in from ._warp_ctc import * File "/home/getalp/desotth/virtenv_pytorch/anaconda3/lib/python3.6/site-packages/warpctc_pytorch/_warp_ctc/init.py", line 3, in from .warp_ctc import lib as _lib, ffi as _ffi ModuleNotFoundError: No module named 'warpctc_pytorch._warp_ctc.warp_ctc'

when running warp-ctc/tests/test.gpu.py

when searching for ModuleNotFoundError: No module named 'warpctc_pytorch._warp_ctc.__warp_ctc, quite some comments appeared, among which to copy contents of directory warp-ctc/pytorch_binding/warpctc_pytorch to anaconda3/lib/python3.6/site-packages...

/anaconda3/lib/python3.6/site-packages adding to path variable

->not working

from setuptools import find_packages find_packages() ['warpctc_pytorch', 'warpctc_pytorch._warp_ctc'

cd /anaconda3/lib/python3.6/site-packages

cd /anaconda3/lib/python3.6/site-packages

python3 import torch from warpctc_pytorch import CTCLoss -> not working

So I would be very grateful with more feedback or help, thanks in advance

[ColorWP]

Hello, About

ImportError: No module named 'warpctc_pytorch._warp_ctc.__warp_ctc'

I have successfully solved this problem, is your Python path problem, You can do it under the terminal,

python3
>>> import warpctc_pytorch
>>> warpctc_pytorch.__path__
['xxx/lib/python3.5/...']     # The real warpctc_pytorch is here!

I hope to help you.

[harshavkumar]

check out, by copying the file:

_warp_ctc.cpython-36m-x86_64-linux-gnu.so

to

warp-ctc/pytorch_binding/warpctc_pytorch/

[MachineVision123]

check out, by copying the file:

_warp_ctc.cpython-36m-x86_64-linux-gnu.so

to

warp-ctc/pytorch_binding/warpctc_pytorch/

Thank harshavkumar . I have solved the problem.

[engreememad]

hello every one I have the same problem, can anyone help me please?

import warpctc_pytorch Traceback (most recent call last): File "", line 1, in File "warpctc_pytorch/init.py", line 6, in from ._warp_ctc import * File "warpctc_pytorch/_warp_ctc/init.py", line 3, in from .__warp_ctc import lib as _lib, ffi as _ffi ImportError: No module named __warp_ctc

I put the WARP_CTC_PATH using this command: export WARP_CTC_PATH="/home/eng-reem/PycharmProjects/pythonProjectpytorch1/warp-ctc/build"

[jdhao]

@engreememad pytorch now has builtin support for ctc loss, you do not really need this anymore.

[engreememad]

@jdhao thank you so much for the replay, I have suffer in the installation of warp-ctc Can I know what is the best version of pytorch can I use it without issues , because I am editing another code have used Warp-ctc for CTCLoss , Can I replace the import sentence just or need another editing for the code?

thanks a lot again for the response and help

[jdhao]

@engreememad you need to try. I am not sure which version of PyTorch works for you. If you do not have a preference, why not use the latest version?

[dzubke]

@engreememad pytorch now has builtin support for ctc loss, you do not really need this anymore.

I have actually found the pytorch ctcloss function to converge 2-4x slower than naren's warpctc implementation. It is definitely easier to use the pytorch ctcloss in some ways, but my experience has been that naren's warpctc implementation converges much faster.

@jdhao - I have found it easiest to use pytorch 1.6 or 1.7 with cuda 9.2 or cuda 10.0.

If you want to use the latest pytorch 1.8, install cuda 10.2 instead of cuda 11. If you do use cuda 11, you'll need to comment out certain lines in the CMakeList.txt file as discussed here: https://github.com/SeanNaren/warp-ctc/issues/185#issuecomment-789440287