j-csc / mlx_bark

Port of Suno's Bark TTS transformer in Apple's MLX Framework
67 stars 2 forks source link

Error audio_arr = codec_decode(fine_tokens) #3

Open padamshrestha opened 8 months ago

padamshrestha commented 8 months ago

Hi,

I'm getting error, is this due to dependencies?

python model.py --text="Hello world!" --path weights/ --model large Generating semantic tokens... 9%|█████████████████████▍ | 70/768 [00:01<00:11, 61.21it/s]Early stop at step 75 with token array([10000], dtype=int32) 10%|██████████████████████▉ | 75/768 [00:01<00:12, 56.71it/s] Generating coarse tokens... 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 4.70it/s] Generating fine tokens... 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 42.83it/s] /Volumes/WD_SN850X_1/Projects/ailgo/.venv/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm. warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.") Traceback (most recent call last): File "/Volumes/WD_SN850X_1/Projects/ailgo/ailgo.python-local/mlx_bark/model.py", line 618, in generate(args.path, args.text, args.model) File "/Volumes/WD_SN850X_1/Projects/ailgo/ailgo.python-local/mlx_bark/model.py", line 604, in generate audio_arr = codec_decode(fine_tokens) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Volumes/WD_SN850X_1/Projects/ailgo/ailgo.python-local/mlx_bark/torch_codec.py", line 23, in codec_decode arr = torch.from_numpy(np.array(fine_tokens, copy=False, dtype=np.int32))[None] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: array(): incompatible function arguments. The following argument types are supported:

  1. (self: mlx.core.array) -> numpy.ndarray

Invoked with: array([[804, 806, 246, ..., 370, 753, 106], [980, 66, 223, ..., 1007, 700, 648], [81, 589, 548, ..., 982, 829, 989], ..., [754, 776, 482, ..., 461, 1005, 76], [762, 569, 162, ..., 711, 755, 464], [768, 978, 829, ..., 534, 472, 16]], dtype=float32), dtype('int32')

j-csc commented 8 months ago

Can't seem to reproduce on my end, looks like a conversion error from mlx -> np. What's your mlx version?

try: pip install -U mlx, lmk

RambleRainbow commented 7 months ago

also encount an error when running ###audio_arr = codec_decode(fine_tokens)

env

Python 3.8.18
Package Version


certifi 2024.2.2 charset-normalizer 3.3.2 einops 0.7.0 encodec 0.1.1 filelock 3.13.1 fsspec 2024.2.0 hf_transfer 0.1.5 huggingface-hub 0.20.3 idna 3.6 Jinja2 3.1.3 MarkupSafe 2.1.5 mlx 0.3.0 mpmath 1.3.0 networkx 3.1 numpy 1.24.4 packaging 23.2 pip 23.3.1 PyYAML 6.0.1 regex 2023.12.25 requests 2.31.0 safetensors 0.4.2 scipy 1.10.1 setuptools 68.2.2 sympy 1.12 tokenizers 0.15.2 torch 2.2.0 torchaudio 2.2.0 tqdm 4.66.2 transformers 4.38.0 typing_extensions 4.9.0 urllib3 2.2.1 wheel 0.41.2

the reports

Generating fine tokens... 100%|███████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 34.66it/s] /opt/homebrew/Caskroom/miniconda/base/envs/barkmlx/lib/python3.8/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm. warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.") zsh: segmentation fault python model.py --path ~/.cache/suno/bark_v0/ --model large --text /opt/homebrew/Caskroom/miniconda/base/envs/barkmlx/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '