Errors when generating-- only with Tortoise

[mqsnn]

Generating tortoise with params: TortoiseParameters( text='Hello world!', voice='daniel,daniel', preset='ultra_fast', seed=-1.0, cvvp_amount=0.0, split_prompt=True, num_autoregressive_samples=4, diffusion_iterations=30, temperature=0.8, length_penalty=1.0, repetition_penalty=2.0, top_p=0.8, max_mel_tokens=500, cond_free=False, cond_free_k=2, diffusion_temperature=1.0, model='Default,Default', name='crazzzzy' ) Traceback (most recent call last): File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction output = await route_utils.call_process_api( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api output = await app.get_blocks().process_api( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api result = await self.call_function( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/blocks.py", line 1199, in call_function prediction = await utils.async_iteration(iterator) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration return await iterator.anext() File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 512, in anext return await anyio.to_thread.run_sync( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, args) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 495, in run_sync_iterator_async return next(iterator) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 649, in gen_wrapper yield from f(args, **kwargs) File "/Users/masong/one-click-installers-tts-6.0/tts-generation-webui/src/tortoise/generation_tab_tortoise.py", line 168, in gen yield from generate_tortoise_long( File "/Users/masong/one-click-installers-tts-6.0/tts-generation-webui/src/tortoise/gen_tortoise.py", line 200, in generate_tortoise_long datas = generate_tortoise( File "/Users/masong/one-click-installers-tts-6.0/tts-generation-webui/src/tortoise/gen_tortoise.py", line 106, in generate_tortoise voice_samples, conditioning_latents = load_voices( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/tortoise/utils/audio.py", line 125, in load_voices clip, latent = load_voice(voice, extra_voice_dirs) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/tortoise/utils/audio.py", line 106, in load_voice paths = voices[voice] KeyError: 'daniel,daniel' { type: 'status', endpoint: '/generate_tortoise_1', fn_index: 251, time: 2024-01-15T21:36:30.843Z, queue: true, message: null, stage: 'error', code: undefined, success: false }

[mqsnn]

I also get sometimes get this error if I don't change anything and just type in a prompt and click generate:

Generating tortoise with params: TortoiseParameters( text='Hello, world!', voice='random', preset='ultra_fast', seed=-1.0, cvvp_amount=0.0, split_prompt=False, num_autoregressive_samples=16, diffusion_iterations=30, temperature=0.8, length_penalty=1.0, repetition_penalty=2.0, top_p=0.8, max_mel_tokens=500, cond_free=False, cond_free_k=2, diffusion_temperature=1.0, model='Default', name='test1' ) Ignored unknown kwarg option normalize Ignored unknown kwarg option normalize Ignored unknown kwarg option normalize Ignored unknown kwarg option normalize Generating autoregressive samples.. 0%| | 0/4 [00:00<?, ?it/s]/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py:823: UserWarning: MPS: no support for int64 repeats mask, casting it to int32 (Triggered internally at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1678454852765/work/aten/src/ATen/native/mps/operations/Repeat.mm:236.) input_ids = input_ids.repeat_interleave(expand_size, dim=0) 0%| | 0/4 [00:00<?, ?it/s] Traceback (most recent call last): File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction output = await route_utils.call_process_api( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api output = await app.get_blocks().process_api( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api result = await self.call_function( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/blocks.py", line 1199, in call_function prediction = await utils.async_iteration(iterator) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration return await iterator.anext() File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 512, in anext return await anyio.to_thread.run_sync( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, args) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 495, in run_sync_iterator_async return next(iterator) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/gradio/utils.py", line 649, in gen_wrapper yield from f(args, kwargs) File "/Users/masong/one-click-installers-tts-6.0/tts-generation-webui/src/tortoise/generation_tab_tortoise.py", line 168, in gen yield from generate_tortoise_long( File "/Users/masong/one-click-installers-tts-6.0/tts-generation-webui/src/tortoise/gen_tortoise.py", line 200, in generate_tortoise_long datas = generate_tortoise( File "/Users/masong/one-click-installers-tts-6.0/tts-generation-webui/src/tortoise/gen_tortoise.py", line 111, in generate_tortoise result, state = tts.tts_with_preset( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/tortoise/api.py", line 321, in tts_with_preset return self.tts(text, settings) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/tortoise/api.py", line 420, in tts codes = autoregressive.inference_speech(auto_conditioning, text_tokens, File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/tortoise/models/autoregressive.py", line 560, in inference_speech gen = self.inference_model.generate(inputs, bos_token_id=self.start_mel_token, pad_token_id=self.stop_mel_token, eos_token_id=self.stop_mel_token, File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py", line 1764, in generate return self.sample( File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py", line 2858, in sample model_inputs = self.prepare_inputs_for_generation(input_ids, model_kwargs) File "/Users/masong/one-click-installers-tts-6.0/installer_files/env/lib/python3.10/site-packages/tortoise/models/autoregressive.py", line 93, in prepare_inputs_for_generation position_ids = attention_mask.long().cumsum(-1) - 1 RuntimeError: MPS does not support cumsum op with int64 input

[rsxdalv]

Thanks, I'll check how to fix it. Gradio likes to break things, but I'll just have to fix them.

[mqsnn]

Thanks, I'll check how to fix it. Gradio likes to break things, but I'll just have to fix them.

Ok, thank you

[rsxdalv]

The first error shouldn't happen with the latest patch.

As for the second error - it's because MPS really works best with the latest pytorch version. I saw people write that if you install the latest pytorch version, it will work. I need to think about how I want to improve the installer for MacOS in general, and what the consequences might be. For now, if you wish I can tell you how to install it, but it has to be done every time after you finish updating.

[mqsnn]

The first error shouldn't happen with the latest patch.

As for the second error - it's because MPS really works best with the latest pytorch version. I saw people write that if you install the latest pytorch version, it will work. I need to think about how I want to improve the installer for MacOS in general, and what the consequences might be. For now, if you wish I can tell you how to install it, but it has to be done every time after you finish updating.

I would appreciate that, sure!

[rsxdalv]

Sorry for not replying. I added automatic torch reinstall at the end of each update, so now after the update, if you had a CPU/MPS torch, it will get updated to one of the latest versions.

If you would like to update to an even newer version such as nightly builds or have other questions, please reopen this issue.