search

k2-fsa / fast_rnnt

A torch implementation of a recursion which turns out to be useful for RNN-T.
Other
136 stars 22 forks source link

missing: CUDNN_LIBRARY_PATH CUDNN_INCLUDE_PATH when installing #13

Closed Anwarvic closed 1 year ago

Anwarvic commented 1 year ago

I'm getting this error when I tried to install Fast RNNT from GitHub repository. These are the commands I used:

$ git clone https://github.com/danpovey/fast_rnnt.git
$ cd fast_rnnt
$ python setup.py install

All needed requirements are met:

The following is the full error trace:

running install
/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
writing fast_rnnt.egg-info/PKG-INFO
writing dependency_links to fast_rnnt.egg-info/dependency_links.txt
writing requirements to fast_rnnt.egg-info/requires.txt
writing top-level names to fast_rnnt.egg-info/top_level.txt
reading manifest file 'fast_rnnt.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
adding license file 'LICENSE'
writing manifest file 'fast_rnnt.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying fast_rnnt/python/fast_rnnt/__init__.py -> build/lib.linux-x86_64-cpython-39/fast_rnnt
running build_ext
For fast compilation, run:
export FT_MAKE_ARGS="-j"; python setup.py install
Setting PYTHON_EXECUTABLE to /gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/bin/PYTHON39
build command is:

            cd build/temp.linux-x86_64-cpython-39

            cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/bin/PYTHON39 /gfs/project/stag/users/manwar/speechbrain_PR/fast_rnnt

            make  _fast_rnnt

-- C++ Standard version: 14
-- Enabled languages: CXX;CUDA
-- Automatic GPU detection failed. Building for common architectures.
-- Autodetected CUDA architecture(s): 3.5;5.0;5.2;6.0;6.1;7.0;7.5;7.5+PTX
-- FT_COMPUTE_ARCH_FLAGS: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_75,code=compute_75
-- FT_COMPUTE_ARCH_CANDIDATES 35;50;60;61;70;75
-- Adding arch 35
-- Adding arch 50
-- Adding arch 60
-- Adding arch 61
-- Adding arch 70
-- Adding arch 75
-- FT_COMPUTE_ARCHS: 35;50;60;61;70;75
-- Downloading pybind11
-- pybind11 is downloaded to /home/manwar/stag/speechbrain_PR/fast_rnnt/build/temp.linux-x86_64-cpython-39/_deps/pybind11-src
-- pybind11 v2.6.0 
-- Python executable: /home/manwar/stag/speechbrain_PR/py39_PR/bin/PYTHON39
-- Caffe2: CUDA detected: 10.2
-- Caffe2: CUDA nvcc is: /nfs/core/cuda/10.2/bin/nvcc
-- Caffe2: CUDA toolkit directory: /nfs/core/cuda/10.2
-- Caffe2: Header version is: 10.2
-- Could NOT find CUDNN (missing: CUDNN_LIBRARY_PATH CUDNN_INCLUDE_PATH) 
CMake Warning at /home/manwar/stag/speechbrain_PR/py39_PR/lib/python3.9/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:111 (message):
  Caffe2: Cannot find cuDNN library.  Turning the option off
Call Stack (most recent call first):
  /home/manwar/stag/speechbrain_PR/py39_PR/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
  /home/manwar/stag/speechbrain_PR/py39_PR/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  cmake/torch.cmake:11 (find_package)
  CMakeLists.txt:135 (include)

-- /nfs/core/cuda/10.2/lib64/libnvrtc.so shorthash is 08c4863f
-- Automatic GPU detection failed. Building for common architectures.
-- Autodetected CUDA architecture(s): 3.5;5.0;5.2;6.0;6.1;7.0;7.5;7.5+PTX
-- Added CUDA NVCC flags for: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_75,code=compute_75
CMake Error at /home/manwar/stag/speechbrain_PR/py39_PR/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:96 (message):
  Your installed Caffe2 version uses cuDNN but I cannot find the cuDNN
  libraries.  Please set the proper cuDNN prefixes and / or install cuDNN.
Call Stack (most recent call first):
  /home/manwar/stag/speechbrain_PR/py39_PR/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  cmake/torch.cmake:11 (find_package)
  CMakeLists.txt:135 (include)

-- Configuring incomplete, errors occurred!
See also "/home/manwar/stag/speechbrain_PR/fast_rnnt/build/temp.linux-x86_64-cpython-39/CMakeFiles/CMakeOutput.log".
See also "/home/manwar/stag/speechbrain_PR/fast_rnnt/build/temp.linux-x86_64-cpython-39/CMakeFiles/CMakeError.log".
make: *** No rule to make target '_fast_rnnt'.  Stop.
Traceback (most recent call last):
  File "/gfs/project/stag/users/manwar/speechbrain_PR/fast_rnnt/setup.py", line 106, in <module>
    setuptools.setup(
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
    return run_commands(dist)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
    dist.run_commands()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
    self.run_command(cmd)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/dist.py", line 1224, in run_command
    super().run_command(command)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/dist.py", line 1224, in run_command
    super().run_command(command)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/dist.py", line 1224, in run_command
    super().run_command(command)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/dist.py", line 1224, in run_command
    super().run_command(command)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
    self.build_extensions()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions
    self._build_extensions_serial()
  File "/gfs/project/stag/users/manwar/speechbrain_PR/py39_PR/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial
    self.build_extension(ext)
  File "/gfs/project/stag/users/manwar/speechbrain_PR/fast_rnnt/setup.py", line 60, in build_extension
    raise Exception(
Exception: 
Build fast_rnnt failed. Please check the error message.
You can ask for help by creating an issue on GitHub.

Click:
    https://github.com/danpovey/fast_rnnt/issues/new
csukuangfj commented 1 year ago

Please install cudnn.

csukuangfj commented 1 year ago

Please first check whether you have installed cudnn or not.

If not, you can use the following commands to install it.

wget https://huggingface.co/csukuangfj/cudnn/resolve/main/cudnn-10.2-linux-x64-v8.0.2.39.tgz
tar xvf cudnn-10.2-linux-x64-v8.0.2.39.tgz --strip-components 1 -C /nfs/core/cuda/10.2
Anwarvic commented 1 year ago

Thank you for your help. My problem was very simple. I wasn't connected to the GPU server where CUDNN is installed.