search

MahmoudAshraf97 / whisper-diarization

Automatic Speech Recognition with Speaker Diarization based on OpenAI Whisper
BSD 2-Clause "Simplified" License
3.28k stars 272 forks source link

Fix Cython 3.0.0 brokes PyYAML==5.4.1 (from NeMo requirements) #73

Closed Ar4ikov closed 6 months ago

Ar4ikov commented 1 year ago

Actual result

Whenever you'd try to install pip install Cython, it brokes the PyYAML==5.4.1 (see https://github.com/yaml/pyyaml/issues/601)

error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [62 lines of output]
      C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\config\setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!

              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.

              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.

              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************

      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running egg_info
      writing lib3\PyYAML.egg-info\PKG-INFO
      writing dependency_links to lib3\PyYAML.egg-info\dependency_links.txt
      writing top-level names to lib3\PyYAML.egg-info\top_level.txt
      Traceback (most recent call last):
        File "C:\Users\bizy1\.conda\envs\pytorch\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\bizy1\.conda\envs\pytorch\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "C:\Users\bizy1\.conda\envs\pytorch\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
          return run_commands(dist)
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
          dist.run_commands()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\dist.py", line 1234, in run_command
          super().run_command(command)
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 314, in run
          self.find_sources()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 322, in find_sources
          mm.run()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 551, in run
          self.add_defaults()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 589, in add_defaults
          sdist.add_defaults(self)
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\command\sdist.py", line 104, in add_defaults
          super().add_defaults()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\_distutils\command\sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\_distutils\command\sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
        File "<string>", line 201, in get_source_files
        File "C:\Users\bizy1\AppData\Local\Temp\pip-build-env-23p_rbwi\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

Expected result

PyYAML==5.4.1 install correctly

Fix

https://github.com/yaml/pyyaml/issues/601#issuecomment-1660180265

full installation of PyYAML looks like this:

pip install "Cython<3" && pip install pyyaml==5.4.1 --no-cache-dir --no-build-isolation
MahmoudAshraf97 commented 1 year ago

The --no-build-isolation solves the problem, you can use it directly when installing the requirements file

zhaoyang-0204 commented 1 year ago

Maybe one want to use ""pip install wheel"" first.

Mzaxd commented 1 year ago

"pip install pybind11" before requirements maybe fix it

dataf3l commented 1 year ago

none of these worked:

10693  pip install -r requirements.txt
10694  pip install wheel
10695  pip install cython
10696  pip install -r requirements.txt
10697  pip install -r requirements.txt --no-build-isolation
10698  pip install --no-build-isolation -r requirements.txt
10699  pip install pybind11
10700  pip install --no-build-isolation -r requirements.txt

despite this:

pip install --no-build-isolation -r requirements.txt
Collecting git+https://github.com/m-bain/whisperX.git@4cb167a225c0ebaea127fd6049abfaa3af9f8bb4 (from -r requirements.txt (line 5))
  Cloning https://github.com/m-bain/whisperX.git (to revision 4cb167a225c0ebaea127fd6049abfaa3af9f8bb4) to /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-req-build-9uj9n27f
  Running command git clone --filter=blob:none --quiet https://github.com/m-bain/whisperX.git /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-req-build-9uj9n27f
  Running command git rev-parse -q --verify 'sha^4cb167a225c0ebaea127fd6049abfaa3af9f8bb4'
  Running command git fetch -q https://github.com/m-bain/whisperX.git 4cb167a225c0ebaea127fd6049abfaa3af9f8bb4
  Running command git checkout -q 4cb167a225c0ebaea127fd6049abfaa3af9f8bb4
  Resolved https://github.com/m-bain/whisperX.git to commit 4cb167a225c0ebaea127fd6049abfaa3af9f8bb4
  Preparing metadata (setup.py) ... done
Collecting demucs
  Cloning https://github.com/facebookresearch/demucs to /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-install-klpzp46v/demucs_f3843e8175de407cad1d76caabbe5d25
  Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/demucs /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-install-klpzp46v/demucs_f3843e8175de407cad1d76caabbe5d25
  Resolved https://github.com/facebookresearch/demucs to commit 404690cb2555d767312cb2cee8a68a6d7bff018d
  Preparing metadata (setup.py) ... done
Collecting wget
  Using cached wget-3.2.zip (10 kB)
  Preparing metadata (setup.py) ... done
Collecting nemo_toolkit[asr]==1.19.1
  Using cached nemo_toolkit-1.19.1-py3-none-any.whl (2.4 MB)
Collecting transformers>=4.26.1
  Using cached transformers-4.32.1-py3-none-any.whl (7.5 MB)
Collecting faster-whisper==0.7.1
  Using cached faster_whisper-0.7.1-py3-none-any.whl (1.5 MB)
Collecting deepmultilingualpunctuation
  Using cached deepmultilingualpunctuation-1.0.1-py3-none-any.whl (5.4 kB)
Collecting huggingface-hub
  Using cached huggingface_hub-0.16.4-py3-none-any.whl (268 kB)
Collecting numba
  Using cached numba-0.57.1-cp311-cp311-macosx_11_0_arm64.whl (2.5 MB)
Collecting numpy>=1.22
  Using cached numpy-1.25.2-cp311-cp311-macosx_11_0_arm64.whl (14.0 MB)
Collecting onnx>=1.7.0
  Using cached onnx-1.14.1-cp311-cp311-macosx_10_12_universal2.whl (15.2 MB)
Collecting python-dateutil
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting ruamel.yaml
  Using cached ruamel.yaml-0.17.32-py3-none-any.whl (112 kB)
Collecting scikit-learn
  Using cached scikit_learn-1.3.0-cp311-cp311-macosx_12_0_arm64.whl (9.4 MB)
Collecting setuptools==65.5.1
  Using cached setuptools-65.5.1-py3-none-any.whl (1.2 MB)
Collecting tensorboard
  Using cached tensorboard-2.14.0-py3-none-any.whl (5.5 MB)
Collecting text-unidecode
  Using cached text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
Collecting torch
  Using cached torch-2.0.1-cp311-none-macosx_11_0_arm64.whl (55.8 MB)
Collecting tqdm>=4.41.0
  Using cached tqdm-4.66.1-py3-none-any.whl (78 kB)
Collecting wrapt
  Using cached wrapt-1.15.0-cp311-cp311-macosx_11_0_arm64.whl (36 kB)
Collecting braceexpand
  Using cached braceexpand-0.1.7-py2.py3-none-any.whl (5.9 kB)
Collecting editdistance
  Using cached editdistance-0.6.2-cp311-cp311-macosx_11_0_arm64.whl (20 kB)
Collecting g2p-en
  Using cached g2p_en-2.1.0-py3-none-any.whl (3.1 MB)
Collecting inflect
  Using cached inflect-7.0.0-py3-none-any.whl (34 kB)
Collecting ipywidgets
  Using cached ipywidgets-8.1.0-py3-none-any.whl (139 kB)
Collecting jiwer
  Using cached jiwer-3.0.2-py3-none-any.whl (21 kB)
Collecting kaldi-python-io
  Using cached kaldi-python-io-1.2.2.tar.gz (8.8 kB)
  Preparing metadata (setup.py) ... done
Collecting kaldiio
  Using cached kaldiio-2.18.0-py3-none-any.whl (28 kB)
Collecting librosa>=0.9.0
  Using cached librosa-0.10.1-py3-none-any.whl (253 kB)
Collecting marshmallow
  Using cached marshmallow-3.20.1-py3-none-any.whl (49 kB)
Collecting matplotlib
  Using cached matplotlib-3.7.2-cp311-cp311-macosx_11_0_arm64.whl (7.3 MB)
Collecting packaging
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting pyannote.core
  Using cached pyannote.core-5.0.0-py3-none-any.whl (58 kB)
Collecting pyannote.metrics
  Using cached pyannote.metrics-3.2.1-py3-none-any.whl (51 kB)
Collecting pydub
  Using cached pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting scipy>=0.14
  Using cached scipy-1.11.2-cp311-cp311-macosx_12_0_arm64.whl (29.6 MB)
Collecting soundfile
  Using cached soundfile-0.12.1-py2.py3-none-macosx_11_0_arm64.whl (1.1 MB)
Collecting sox
  Using cached sox-1.4.1-py2.py3-none-any.whl (39 kB)
Collecting texterrors
  Using cached texterrors-0.4.4.tar.gz (20 kB)
  Preparing metadata (setup.py) ... done
Collecting hydra-core<1.3,>=1.2.0
  Using cached hydra_core-1.2.0-py3-none-any.whl (151 kB)
Collecting omegaconf<2.3,>=2.2
  Using cached omegaconf-2.2.3-py3-none-any.whl (79 kB)
Collecting pytorch-lightning<=1.9.4,>=1.9.0
  Using cached pytorch_lightning-1.9.4-py3-none-any.whl (827 kB)
Collecting pyyaml<6
  Using cached PyYAML-5.4.1.tar.gz (175 kB)
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [57 lines of output]
      /Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/config/setupcfg.py:516: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running dist_info
      creating /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-modern-metadata-mdnccoim/PyYAML.egg-info
      writing /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-modern-metadata-mdnccoim/PyYAML.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-modern-metadata-mdnccoim/PyYAML.egg-info/dependency_links.txt
      writing top-level names to /private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-modern-metadata-mdnccoim/PyYAML.egg-info/top_level.txt
      writing manifest file '/private/var/folders/yv/lystpk8n2015cf8vmqd2yj_c0000gp/T/pip-modern-metadata-mdnccoim/PyYAML.egg-info/SOURCES.txt'
      Traceback (most recent call last):
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 377, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/__init__.py", line 108, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/dist.py", line 1221, in run_command
          super().run_command(command)
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/command/dist_info.py", line 98, in run
          self.egg_info.run()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 318, in run
          self.find_sources()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 326, in find_sources
          mm.run()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 570, in run
          self.add_defaults()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 608, in add_defaults
          sdist.add_defaults(self)
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/command/sdist.py", line 106, in add_defaults
          super().add_defaults()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/Users/b/study/ml/whisper-diarization/venv/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip

so the problem is not solved by any of the commands mentioned on this chat window unfortunately

dataf3l commented 1 year ago

these two worked at the end:

10703  pip install "Cython<3" && pip install pyyaml==5.4.1 --no-cache-dir --no-build-isolation
10704  pip install --no-build-isolation -r requirements.txt

no idea why it worked...