bene-ges / nemo_compatible

useful things that work with NVIDIA NeMo library
Apache License 2.0
9 stars 1 forks source link

Error at startup test.sh and other startup files #19

Closed Vubni closed 4 months ago

Vubni commented 4 months ago

I may not have framed the title exactly (forgive me for that), but I meant that any file for starting test or training causes me the same error.

Here is the error I get when trying to run test.sh from ru_ipa_fastpitch_hifigan:

./test.sh
./test.sh: строка 2: conda: команда не найдена
fatal: целевой путь «ru_g2p_ipa_bert_large» уже существует и не является пустым каталогом.
fatal: целевой путь «tts_ru_ipa_fastpitch_ruslan» уже существует и не является пустым каталогом.
fatal: целевой путь «tts_ru_hifigan_ruslan» уже существует и не является пустым каталогом.
Traceback (most recent call last):
  File "NeMo/examples/nlp/text_normalization_as_tagging/normalization_as_tagging_infer.py", line 42, in <module>
    from helpers import ITN_MODEL, instantiate_model_and_trainer
  File "/home/egor/synthesys/NeMo/examples/nlp/text_normalization_as_tagging/helpers.py", line 22, in <module>
    from nemo.collections.nlp.models import ThutmoseTaggerModel
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/nlp/__init__.py", line 15, in <module>
    from nemo.collections.nlp import data, losses, models, modules
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/nlp/data/__init__.py", line 17, in <module>
    from nemo.collections.nlp.data.information_retrieval.information_retrieval_dataset import (
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/nlp/data/information_retrieval/__init__.py", line 15, in <module>
    from nemo.collections.nlp.data.information_retrieval.information_retrieval_dataset import (
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/nlp/data/information_retrieval/information_retrieval_dataset.py", line 24, in <module>
    from nemo.collections.common.tokenizers.tokenizer_spec import TokenizerSpec
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/__init__.py", line 16, in <module>
    from nemo.collections.common import data, losses, parts, tokenizers
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/tokenizers/__init__.py", line 17, in <module>
    from nemo.collections.common.tokenizers.canary_tokenizer import CanaryTokenizer
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/tokenizers/canary_tokenizer.py", line 51, in <module>
    class CanaryTokenizer(AggregateTokenizer):
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/tokenizers/canary_tokenizer.py", line 105, in CanaryTokenizer
    def build_special_tokenizer(output_dir: str | Path) -> SentencePieceTokenizer:
TypeError: unsupported operand type(s) for |: 'type' and 'type'
Traceback (most recent call last):
  File "nemo_compatible/scripts/tts/ru_g2p_ipa/preprocess_text_before_tts.py", line 30, in <module>
    with open(args.g2p_name, "r", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test_input.txt.words.g2p'
Traceback (most recent call last):
  File "nemo_compatible/scripts/tts/tts_infer.py", line 6, in <module>
    from nemo.collections.tts.models.base import SpectrogramGenerator, Vocoder
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/tts/__init__.py", line 15, in <module>
    from nemo.collections.tts import data, losses, models, modules
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/tts/models/__init__.py", line 16, in <module>
    from nemo.collections.tts.models.audio_codec import AudioCodecModel
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/tts/models/audio_codec.py", line 27, in <module>
    from nemo.collections.tts.losses.audio_codec_loss import (
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/tts/losses/audio_codec_loss.py", line 21, in <module>
    from nemo.collections.asr.parts.preprocessing.features import FilterbankFeatures
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/asr/__init__.py", line 15, in <module>
    from nemo.collections.asr import data, losses, models, modules
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/asr/losses/__init__.py", line 16, in <module>
    from nemo.collections.asr.losses.audio_losses import SDRLoss
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/asr/losses/audio_losses.py", line 21, in <module>
    from nemo.collections.asr.parts.preprocessing.features import make_seq_mask_like
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/asr/parts/preprocessing/__init__.py", line 16, in <module>
    from nemo.collections.asr.parts.preprocessing.features import FeaturizerFactory, FilterbankFeatures, WaveformFeaturizer
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/asr/parts/preprocessing/features.py", line 44, in <module>
    from nemo.collections.asr.parts.preprocessing.perturb import AudioAugmentor
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/asr/parts/preprocessing/perturb.py", line 50, in <module>
    from nemo.collections.common.parts.preprocessing import collections, parsers
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/__init__.py", line 16, in <module>
    from nemo.collections.common import data, losses, parts, tokenizers
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/tokenizers/__init__.py", line 17, in <module>
    from nemo.collections.common.tokenizers.canary_tokenizer import CanaryTokenizer
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/tokenizers/canary_tokenizer.py", line 51, in <module>
    class CanaryTokenizer(AggregateTokenizer):
  File "/home/egor/.local/lib/python3.8/site-packages/nemo/collections/common/tokenizers/canary_tokenizer.py", line 105, in CanaryTokenizer
    def build_special_tokenizer(output_dir: str | Path) -> SentencePieceTokenizer:
TypeError: unsupported operand type(s) for |: 'type' and 'type'

My system: ubuntu 22.04.04 Python: 3.8 Pip list:

Package                      Version
---------------------------- ----------------
absl-py                      2.1.0
aiohttp                      3.9.3
aiosignal                    1.3.1
annotated-types              0.6.0
antlr4-python3-runtime       4.9.3
apturl                       0.5.2
asgiref                      3.7.2
astunparse                   1.6.3
async-timeout                4.0.3
attrs                        23.2.0
audioread                    3.0.1
bcrypt                       3.2.0
blinker                      1.4
Brlapi                       0.8.3
cachetools                   5.3.3
certifi                      2020.6.20
cffi                         1.16.0
chardet                      4.0.0
charset-normalizer           3.3.2
click                        8.0.3
colorama                     0.4.4
command-not-found            0.3
contourpy                    1.1.1
cryptography                 41.0.7
cupshelpers                  1.0
cycler                       0.12.1
dbus-python                  1.2.18
decorator                    5.1.1
defer                        1.0.6
distro                       1.7.0
distro-info                  1.1+ubuntu0.2
Django                       3.2.23
django-auditlog              2.3.0
django-filter                23.5
django-js-asset              2.2.0
django-mptt                  0.14.0
djangorestframework          3.14.0
drf-excel                    2.4.0
drf-flex-fields              1.0.2
duplicity                    0.8.21
einops                       0.7.0
et-xmlfile                   1.1.0
fasteners                    0.14.1
filelock                     3.13.1
flatbuffers                  24.3.7
fonttools                    4.50.0
frozenlist                   1.4.1
fsspec                       2024.3.0
future                       0.18.2
gast                         0.4.0
google-auth                  2.28.2
google-auth-oauthlib         1.0.0
google-pasta                 0.2.0
grpcio                       1.62.1
h5py                         3.10.0
helpers                      0.2.0
httplib2                     0.20.2
huggingface-hub              0.21.4
Hydra                        2.5
hydra-core                   1.3.2
idna                         3.3
importlib-metadata           4.6.4
importlib_resources          6.3.1
jeepney                      0.7.1
Jinja2                       3.1.3
joblib                       1.3.2
keras                        2.13.1
keyring                      23.5.0
kiwisolver                   1.4.5
language-selector            0.1
launchpadlib                 1.10.16
lazr.restfulclient           0.14.4
lazr.uri                     1.0.6
lazy_loader                  0.3
ldap3                        2.9.1
libclang                     18.1.1
librosa                      0.10.1
lightning-utilities          0.10.1
llvmlite                     0.41.1
lockfile                     0.12.2
louis                        3.20.0
macaroonbakery               1.3.1
Mako                         1.1.3
Markdown                     3.6
MarkupSafe                   2.1.5
matplotlib                   3.7.5
monotonic                    1.6
more-itertools               8.10.0
mpmath                       1.3.0
msgpack                      1.0.8
multidict                    6.0.5
NEMO                         5.2.0
nemo_toolkit                 1.23.0
netifaces                    0.11.0
networkx                     3.1
numba                        0.58.1
numpy                        1.24.3
nvidia-cublas-cu12           12.1.3.1
nvidia-cuda-cupti-cu12       12.1.105
nvidia-cuda-nvrtc-cu12       12.1.105
nvidia-cuda-runtime-cu12     12.1.105
nvidia-cudnn-cu12            8.9.2.26
nvidia-cufft-cu12            11.0.2.54
nvidia-curand-cu12           10.3.2.106
nvidia-cusolver-cu12         11.4.5.107
nvidia-cusparse-cu12         12.1.0.106
nvidia-nccl-cu12             2.19.3
nvidia-nvjitlink-cu12        12.4.99
nvidia-nvtx-cu12             12.1.105
oauthlib                     3.2.0
olefile                      0.46
omegaconf                    2.3.0
onnx                         1.15.0
openpyxl                     3.1.2
opt-einsum                   3.3.0
packaging                    24.0
paramiko                     2.9.3
pexpect                      4.8.0
pillow                       10.2.0
pip                          24.0
platformdirs                 4.2.0
pooch                        1.8.1
protobuf                     4.25.3
ptyprocess                   0.7.0
pyasn1                       0.5.1
pyasn1-modules               0.3.0
pybind11                     2.11.1
pycairo                      1.20.1
pycparser                    2.21
pycups                       2.0.1
pydantic                     1.10.14
pydantic_core                2.16.3
PyGObject                    3.42.1
PyJWT                        2.3.0
pymacaroons                  0.13.0
pymodbus                     3.3.2
PyNaCl                       1.5.0
pyparsing                    2.4.7
pyRFC3339                    1.1
python-apt                   2.4.0+ubuntu3
python-dateutil              2.8.2
python-debian                0.1.43+ubuntu1.1
pytorch-lightning            2.2.1
pytz                         2023.3
pyxdg                        0.27
PyYAML                       5.4.1
regex                        2023.12.25
reportlab                    3.6.8
requests                     2.31.0
requests-oauthlib            1.4.0
rsa                          4.9
ruamel.yaml                  0.18.6
ruamel.yaml.clib             0.2.8
safetensors                  0.4.2
scikit-learn                 1.3.2
scipy                        1.10.1
screen-resolution-extra      0.0.0
SecretStorage                3.3.1
sentencepiece                0.2.0
setuptools                   69.2.0
six                          1.16.0
soundfile                    0.12.1
sox                          1.4.1
soxr                         0.3.7
sqlparse                     0.4.4
sympy                        1.12
systemd-python               234
tensorboard                  2.13.0
tensorboard-data-server      0.7.2
tensorflow                   2.13.1
tensorflow-addons            0.21.0
tensorflow-estimator         2.13.0
tensorflow-hub               0.16.1
tensorflow-io                0.34.0
tensorflow-io-gcs-filesystem 0.34.0
tensorflow-text              2.13.0
termcolor                    2.4.0
text-unidecode               1.3
tf-keras                     2.15.0
threadpoolctl                3.3.0
tokenizers                   0.15.2
torch                        2.2.1
torchmetrics                 1.3.2
tqdm                         4.66.2
transformers                 4.38.2
triton                       2.2.0
typeguard                    2.13.3
typing_extensions            4.5.0
ubuntu-drivers-common        0.0.0
ubuntu-pro-client            8001
ufw                          0.36.1
unattended-upgrades          0.1
urllib3                      1.26.5
usb-creator                  0.3.7
wadllib                      1.3.6
Werkzeug                     3.0.1
wget                         3.2
wheel                        0.43.0
wrapt                        1.16.0
xdg                          5
xkit                         0.0.0
yarl                         1.9.4
zipp                         3.18.1
bene-ges commented 4 months ago

Hi, @Vubni I suppose that python 3.8 is incompatible with latest nemo version - it requires 3.10. Could you try Nemo 1.21 or lower?

Vubni commented 4 months ago

It really helped, thanks!