search

maxrmorrison / torbi

Viterbi decoding in PyTorch
MIT License
27 stars 2 forks source link

Install need g++-11 #2

Open lixuyuan102 opened 3 months ago

lixuyuan102 commented 3 months ago

When I installed the torbi, I got an error of "subprocess.CalledProcessError: Command '['which', 'g++-11']' returned non-zero exit status 1". Is the g++-11 necessary for torbi?

lixuyuan102 commented 3 months ago

Here is the log:

Processing ./torbi Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: numpy in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from torbi==0.0.1) (1.23.5) Collecting torch==1.12.0 (from torbi==0.0.1) Using cached http://pypi.speech.local/packages/86/c3/30eb447a38bb73d57883ec0941e213249b2001d78332a3026351e0ee8d1f/torch-1.12.0-cp38-cp38-manylinux1_x86_64.whl (776.3 MB) Requirement already satisfied: torchutil in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from torbi==0.0.1) (0.0.13) Requirement already satisfied: yapecs in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from torbi==0.0.1) (0.0.8) Requirement already satisfied: typing-extensions in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from torch==1.12.0->torbi==0.0.1) (4.12.2) Requirement already satisfied: apprise in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from torchutil->torbi==0.0.1) (1.8.1) Requirement already satisfied: tensorboard in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from torchutil->torbi==0.0.1) (2.14.0) Requirement already satisfied: filelock in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from yapecs->torbi==0.0.1) (3.15.4) Requirement already satisfied: certifi in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from apprise->torchutil->torbi==0.0.1) (2024.7.4) Requirement already satisfied: requests in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from apprise->torchutil->torbi==0.0.1) (2.32.3) Requirement already satisfied: requests-oauthlib in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from apprise->torchutil->torbi==0.0.1) (2.0.0) Requirement already satisfied: click>=5.0 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from apprise->torchutil->torbi==0.0.1) (8.1.7) Requirement already satisfied: markdown in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from apprise->torchutil->torbi==0.0.1) (3.6) Requirement already satisfied: PyYAML in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from apprise->torchutil->torbi==0.0.1) (6.0.2) Requirement already satisfied: absl-py>=0.4 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (2.1.0) Requirement already satisfied: grpcio>=1.48.2 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (1.65.4) Requirement already satisfied: google-auth<3,>=1.6.3 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (2.33.0) Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (1.0.0) Requirement already satisfied: protobuf>=3.19.6 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (5.27.3) Requirement already satisfied: setuptools>=41.0.0 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (72.1.0) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (3.0.3) Requirement already satisfied: wheel>=0.26 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from tensorboard->torchutil->torbi==0.0.1) (0.43.0) Requirement already satisfied: cachetools<6.0,>=2.0.0 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard->torchutil->torbi==0.0.1) (5.4.0) Requirement already satisfied: pyasn1-modules>=0.2.1 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard->torchutil->torbi==0.0.1) (0.4.0) Requirement already satisfied: rsa<5,>=3.1.4 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard->torchutil->torbi==0.0.1) (4.9) Requirement already satisfied: importlib-metadata>=4.4 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from markdown->apprise->torchutil->torbi==0.0.1) (4.13.0) Requirement already satisfied: charset-normalizer<4,>=2 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from requests->apprise->torchutil->torbi==0.0.1) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from requests->apprise->torchutil->torbi==0.0.1) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from requests->apprise->torchutil->torbi==0.0.1) (2.2.2) Requirement already satisfied: oauthlib>=3.0.0 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from requests-oauthlib->apprise->torchutil->torbi==0.0.1) (3.2.2) Requirement already satisfied: MarkupSafe>=2.1.1 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from werkzeug>=1.0.1->tensorboard->torchutil->torbi==0.0.1) (2.1.5) Requirement already satisfied: zipp>=0.5 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from importlib-metadata>=4.4->markdown->apprise->torchutil->torbi==0.0.1) (3.19.2) Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->torchutil->torbi==0.0.1) (0.6.0) Building wheels for collected packages: torbi Building wheel for torbi (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for torbi (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [170 lines of output] running bdist_wheel running build running build_py creating build/lib.linux-x86_64-cpython-38 creating build/lib.linux-x86_64-cpython-38/torbi copying torbi/main.py -> build/lib.linux-x86_64-cpython-38/torbi copying torbi/init.py -> build/lib.linux-x86_64-cpython-38/torbi copying torbi/chunk.py -> build/lib.linux-x86_64-cpython-38/torbi copying torbi/core.py -> build/lib.linux-x86_64-cpython-38/torbi creating build/lib.linux-x86_64-cpython-38/torbi/reference copying torbi/reference/init.py -> build/lib.linux-x86_64-cpython-38/torbi/reference copying torbi/reference/core.py -> build/lib.linux-x86_64-cpython-38/torbi/reference creating build/lib.linux-x86_64-cpython-38/torbi/partition copying torbi/partition/main.py -> build/lib.linux-x86_64-cpython-38/torbi/partition copying torbi/partition/init.py -> build/lib.linux-x86_64-cpython-38/torbi/partition copying torbi/partition/core.py -> build/lib.linux-x86_64-cpython-38/torbi/partition creating build/lib.linux-x86_64-cpython-38/torbi/data copying torbi/data/dataset.py -> build/lib.linux-x86_64-cpython-38/torbi/data copying torbi/data/init.py -> build/lib.linux-x86_64-cpython-38/torbi/data copying torbi/data/collate.py -> build/lib.linux-x86_64-cpython-38/torbi/data copying torbi/data/loader.py -> build/lib.linux-x86_64-cpython-38/torbi/data creating build/lib.linux-x86_64-cpython-38/torbi/config copying torbi/config/init.py -> build/lib.linux-x86_64-cpython-38/torbi/config copying torbi/config/static.py -> build/lib.linux-x86_64-cpython-38/torbi/config copying torbi/config/defaults.py -> build/lib.linux-x86_64-cpython-38/torbi/config creating build/lib.linux-x86_64-cpython-38/torbi/evaluate copying torbi/evaluate/main.py -> build/lib.linux-x86_64-cpython-38/torbi/evaluate copying torbi/evaluate/init.py -> build/lib.linux-x86_64-cpython-38/torbi/evaluate copying torbi/evaluate/metrics.py -> build/lib.linux-x86_64-cpython-38/torbi/evaluate copying torbi/evaluate/core.py -> build/lib.linux-x86_64-cpython-38/torbi/evaluate creating build/lib.linux-x86_64-cpython-38/torbi/data/preprocess copying torbi/data/preprocess/main.py -> build/lib.linux-x86_64-cpython-38/torbi/data/preprocess copying torbi/data/preprocess/init.py -> build/lib.linux-x86_64-cpython-38/torbi/data/preprocess copying torbi/data/preprocess/core.py -> build/lib.linux-x86_64-cpython-38/torbi/data/preprocess creating build/lib.linux-x86_64-cpython-38/torbi/data/download copying torbi/data/download/main.py -> build/lib.linux-x86_64-cpython-38/torbi/data/download copying torbi/data/download/init.py -> build/lib.linux-x86_64-cpython-38/torbi/data/download copying torbi/data/download/core.py -> build/lib.linux-x86_64-cpython-38/torbi/data/download running egg_info writing torbi.egg-info/PKG-INFO writing dependency_links to torbi.egg-info/dependency_links.txt writing requirements to torbi.egg-info/requires.txt writing top-level names to torbi.egg-info/top_level.txt reading manifest file 'torbi.egg-info/SOURCES.txt' adding license file 'LICENSE' writing manifest file 'torbi.egg-info/SOURCES.txt' copying torbi/viterbi.cpp -> build/lib.linux-x86_64-cpython-38/torbi copying torbi/viterbi_kernel.cu -> build/lib.linux-x86_64-cpython-38/torbi creating build/lib.linux-x86_64-cpython-38/torbi/assets creating build/lib.linux-x86_64-cpython-38/torbi/assets/partitions copying torbi/assets/partitions/daps.json -> build/lib.linux-x86_64-cpython-38/torbi/assets/partitions copying torbi/assets/partitions/vctk.json -> build/lib.linux-x86_64-cpython-38/torbi/assets/partitions creating build/lib.linux-x86_64-cpython-38/torbi/assets/stats copying torbi/assets/stats/transition.pt -> build/lib.linux-x86_64-cpython-38/torbi/assets/stats running build_ext /tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for [tool.setuptools] in pyproject.toml is still beta. warnings.warn(msg, _BetaConfiguration) /tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.')) /tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'torbi.assets.partitions' as data is deprecated, please list it in packages. !!

      ############################
      # Package would be ignored #
      ############################
      Python recognizes 'torbi.assets.partitions' as an importable package,
      but it is not listed in the `packages` configuration of setuptools.

      'torbi.assets.partitions' has been automatically added to the distribution only
      because it may contain data files, but this behavior is likely to change
      in future versions of setuptools (and therefore is considered deprecated).

      Please make sure that 'torbi.assets.partitions' is included as a package by using
      the `packages` configuration field or the proper discovery methods
      (for example by using `find_namespace_packages(...)`/`find_namespace:`
      instead of `find_packages(...)`/`find:`).

      You can read more about "package discovery" and "data files" on setuptools
      documentation page.

  !!

    check.warn(importable)
  /tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'torbi.assets.stats' as data is deprecated, please list it in `packages`.
      !!

      ############################
      # Package would be ignored #
      ############################
      Python recognizes 'torbi.assets.stats' as an importable package,
      but it is not listed in the `packages` configuration of setuptools.

      'torbi.assets.stats' has been automatically added to the distribution only
      because it may contain data files, but this behavior is likely to change
      in future versions of setuptools (and therefore is considered deprecated).

      Please make sure that 'torbi.assets.stats' is included as a package by using
      the `packages` configuration field or the proper discovery methods
      (for example by using `find_namespace_packages(...)`/`find_namespace:`
      instead of `find_packages(...)`/`find:`).

      You can read more about "package discovery" and "data files" on setuptools
      documentation page.

  !!

    check.warn(importable)
  Traceback (most recent call last):
    File "/data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 413, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 398, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
      exec(code, locals())
    File "<string>", line 24, in <module>
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 108, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1221, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-pklc10nt/normal/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1221, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1221, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run
      _build_ext.run(self)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 420, in build_extensions
      compiler_name, compiler_version = self._check_abi()
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 791, in _check_abi
      _, version = get_compiler_abi_compatibility_and_version(compiler)
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 324, in get_compiler_abi_compatibility_and_version
      if not check_compiler_ok_for_platform(compiler):
    File "/tmp/pip-build-env-pklc10nt/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 280, in check_compiler_ok_for_platform
      which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT)
    File "/data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/subprocess.py", line 411, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/data2/ASR/lixuyuan/anaconda3/envs/vc/lib/python3.8/subprocess.py", line 512, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['which', 'g++-11']' returned non-zero exit status 1.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for torbi Failed to build torbi ERROR: Could not build wheels for torbi, which is required to install pyproject.toml-based projects

maxrmorrison commented 1 month ago

We used g++-11 for development and benchmarking purposes. I have also successfully run torbi using g++ version 9.4.0, so other versions produce correct results—but we have not benchmarked them. If you want to try a different compiler, you can change the compiler executable in setup.py to your desired compiler (e.g., change g++-11 to g++).

lixuyuan102 commented 1 month ago

We used g++-11 for development and benchmarking purposes. I have also successfully run torbi using g++ version 9.4.0, so other versions produce correct results—but we have not benchmarked them. If you want to try a different compiler, you can change the compiler executable in setup.py to your desired compiler (e.g., change g++-11 to g++).

Thanks, I have issued this using a docker image with g++-11, but I'll follow your advice on the local server.