coqui-ai / TTS

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
http://coqui.ai
Mozilla Public License 2.0
35.74k stars 4.38k forks source link

[Bug] Bark does not work out of the box - invalid load key #3804

Open jooray opened 5 months ago

jooray commented 5 months ago

Describe the bug

When I do fresh install of TTS and want to use bark model, it does not work, gives error:

(.venv) user@server : ~/tmp/tts$ tts --model_name 'tts_models/multilingual/multi-dataset/bark' --out_path test.wav --text 'We should see how this works, probably very well.'
 > tts_models/multilingual/multi-dataset/bark is already downloaded.
 > Using model: bark
/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
WARNING:TTS.tts.layers.bark.load_model:found outdated text model, removing...
/Users/user/Library/Application Support/tts/tts_models--multilingual--multi-dataset--bark/text_2.pt
100%|█████████████████████████████████████| 17.1k/17.1k [00:00<00:00, 20.9MiB/s]
Traceback (most recent call last):
  File "/Users/user/tmp/tts/.venv/bin/tts", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/TTS/bin/synthesize.py", line 423, in main
    synthesizer = Synthesizer(
                  ^^^^^^^^^^^^
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/TTS/utils/synthesizer.py", line 109, in __init__
    self._load_tts_from_dir(model_dir, use_cuda)
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/TTS/utils/synthesizer.py", line 164, in _load_tts_from_dir
    self.tts_model.load_checkpoint(config, checkpoint_dir=model_dir, eval=True)
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/TTS/tts/models/bark.py", line 281, in load_checkpoint
    self.load_bark_models()
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/TTS/tts/models/bark.py", line 50, in load_bark_models
    self.semantic_model, self.config = load_model(
                                       ^^^^^^^^^^^
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/TTS/tts/layers/bark/load_model.py", line 125, in load_model
    checkpoint = torch.load(ckpt_path, map_location=device)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/torch/serialization.py", line 1040, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/tmp/tts/.venv/lib/python3.11/site-packages/torch/serialization.py", line 1262, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_pickle.UnpicklingError: invalid load key, '<'.

To Reproduce

macOS 14.5 python version: Python 3.11.9 (main, Apr 2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin

python3.11 -m venv create .venv
. .venv/bin/activate
pip install TTS
tts --model_name 'tts_models/multilingual/multi-dataset/bark' --out_path test.wav --text 'We should see how this works, probably very well.'

python version: Python 3.11.9 (main, Apr 2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin

Expected behavior

No response

Logs

No response

Environment

{
    "CUDA": {
        "GPU": [],
        "available": false,
        "version": null
    },
    "Packages": {
        "PyTorch_debug": false,
        "PyTorch_version": "2.3.1",
        "TTS": "0.22.0",
        "numpy": "1.26.4"
    },
    "System": {
        "OS": "Darwin",
        "architecture": [
            "64bit",
            ""
        ],
        "processor": "arm",
        "python": "3.11.9",
        "version": "Darwin Kernel Version 23.5.0: Wed May  1 20:14:38 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6020"
    }
}


### Additional context

_No response_
eginhard commented 5 months ago

This has been fixed in our fork, you can install it with pip install coqui-tts

stale[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. You might also look our discussion channels.

michaelmior commented 3 months ago

I have the same issue. Also, the Bark config has hardcoded paths to /root which I had to manually edit before I got this far.

stale[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. You might also look our discussion channels.