Expected String and got Path

ObscenityIB commented 3 months ago

Describe the bug Tried this a few months ago and had no luck, tried it again today and still no luck. After installation, generating a test message fails immediately as python seems to not be accepting the output path, or any path for that matter.

To Reproduce Use atsetup.sh to install alltalk standalone. Activate the environment with start_environment.sh. Use atsetup.sh to install DeepSpeed.* Run start_alltalk.sh. Type in some example text and hit Generate.

Text/logs

$ ./start_alltalk.sh 
[AllTalk Startup]     _    _ _ _____     _ _       _____ _____ ____  
[AllTalk Startup]    / \  | | |_   _|_ _| | | __  |_   _|_   _/ ___| 
[AllTalk Startup]   / _ \ | | | | |/ _` | | |/ /    | |   | | \___ \ 
[AllTalk Startup]  / ___ \| | | | | (_| | |   <     | |   | |  ___) |
[AllTalk Startup] /_/   \_\_|_| |_|\__,_|_|_|\_\    |_|   |_| |____/ 
[AllTalk Startup]
[AllTalk Startup] Config file check      : No Updates required
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  NVIDIA Inference is only supported on Ampere and newer architectures
 [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
 [WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[AllTalk Startup] AllTalk startup Mode   : Standalone mode
[AllTalk Startup] WAV file deletion      : Disabled
[AllTalk Startup] DeepSpeed version      : 0.14.3 
[AllTalk Startup] Model is available     : Checking
[AllTalk Startup] Model is available     : Checked
[AllTalk Startup] Current Python Version : 3.11.9
[AllTalk Startup] Current PyTorch Version: 2.3.1+cu121
[AllTalk Startup] Current CUDA Version   : 12.1
[AllTalk Startup] Current TTS Version    : 0.22.0
[AllTalk Startup] Current TTS Version is : Up to date
[AllTalk Startup] AllTalk Github updated : 6th June 2024 at 22:23
[AllTalk Startup] TTS Subprocess         : Starting up
[AllTalk Startup]
[AllTalk Startup] AllTalk Settings & Documentation: http://127.0.0.1:7851
[AllTalk Startup]
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  NVIDIA Inference is only supported on Ampere and newer architectures
 [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
 [WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
[AllTalk Model] XTTSv2 Local Loading xttsv2_2.0.2 into cuda 
[AllTalk Model] Coqui Public Model License
[AllTalk Model] https://coqui.ai/cpml.txt
[AllTalk Model] Model Loaded in 17.14 seconds.
[AllTalk Model] Ready
[AllTalk TTSGen] this is a test, it is only a test
An error occurred: torchaudio_sox::save_audio_file() Expected a value of type 'str' for argument '_0' but instead found type 'PosixPath'.
Position: 0
Value: PosixPath('/home/obscenity/DEV/git/alltalk_tts/outputs/demo_output.wav')
Declaration: torchaudio_sox::save_audio_file(str _0, Tensor _1, int _2, bool _3, float? _4, str? _5, str? _6, int? _7) -> ()
Cast error details: Unable to cast Python instance of type <class 'pathlib.PosixPath'> to C++ type '?' (#define PYBIND11_DETAILED_ERROR_MESSAGES or compile in debug mode for details)
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/responses.py", line 326, in __call__
    stat_result = await anyio.to_thread.run_sync(os.stat, self.path)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/obscenity/DEV/git/alltalk_tts/outputs/undefined'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
    await response(scope, receive, send)
  File "/home/obscenity/DEV/git/alltalk_tts/alltalk_environment/env/lib/python3.11/site-packages/starlette/responses.py", line 329, in __call__
    raise RuntimeError(f"File at path {self.path} does not exist.")
RuntimeError: File at path /home/obscenity/DEV/git/alltalk_tts/outputs/undefined does not exist.

Desktop (please complete the following information): AllTalk was updated: 14/06/2024 Custom Python environment: no Text-generation-webUI was updated: N/A

Additional context * Unrelated, but enabling deepspeed afterwards causes cuda compilation errors, and alltalk will not start if enabled, as well as creating a random log file named '=2.2.1+cu121' in the working directory when installed.

diagnostics.log

erew123 commented 3 months ago

Hi @ObscenityIB

From your screenshot, best I can tell you have installed this as a Standalone? But you aren't using AllTalks custom built Python Environment aka starting AllTalk with start_alltalk.sh?

I can see in the start-up its saying PyTorch 2.3.1:

[AllTalk Startup] Current PyTorch Version: 2.3.1+cu121

Which will probably have issues to be honest with you. Many of the older requirements may not like PyTorch 2.3.x

You should preferably use atsetup.sh to build the custom Python environment, as that builds a tested environment (It will be Pytorch 2.2.x). And then you should also use start_alltalk.sh which loads in the Python environment before starting AllTalk.

That aside, I have also released specific wheel files for DeepSpeed on Linux that will work with the AllTalk custom Python environment details here

That aside, the BETA of AllTalk v2, that installation is further simplified as you install it as a Standalone (building the AllTalk Python Environment) which will auto install DeepSpeed pre-compiled for Linux. https://github.com/erew123/alltalk_tts/tree/alltalkbeta

Thanks

ObscenityIB commented 3 months ago

Hi @erew123, thanks for getting back so quickly.

best I can tell you have installed this as a Standalone?

Yes, I installed it as standalone.

But you aren't using AllTalks custom built Python Environment aka starting AllTalk with start_alltalk.sh?

I am, you can see that at the start of the log output ($ ./start_alltalk.sh), and in the steps to reproduce the issue.

You should preferably use atsetup.sh to build the custom Python environment

This is exactly how I have installed alltalk, running atsetup.sh, again listed in my steps to reproduce.

Thanks for the other notes on DeepSpeed and the pytorch version, I'll try out that beta now.

erew123 commented 3 months ago

@ObscenityIB Theres only 1x way I can think you may have ended up with a PyTorch 2.3.x environment then (on AllTalk v1). The pip application that Python uses to download packages will cache copies of software it downloads. If you have downloaded a version of some software before (in this case PyTorch) and you then attempt another download of the same software, but a different version, even though you explicitly say "download X version" it will just re-use the previously downloaded version.... putting that another way.....

If your computer downloaded PyTorch 2.3.1 for some other software, PIP keeps a locally cached installer for 2.3.1. When the AllTalk v1 setup goes to install PyTorch 2.2,1, pip just goes "here you go, use 2.3.1", which is very frustrating.

You can wipe the pip cache by running pip cache purge https://pip.pypa.io/en/stable/cli/pip_cache/ which wipes all downloaded installers and forces it to re-download everything afresh next time around.

Saying that, the v2 beta doesnt use pip to install Pytorch, but conda install instead, which should mean any cached versions of Pytorch shouldnt matter.

ObscenityIB commented 3 months ago

I've never had anything installed via pip, since using pip is outdated and actively prevented by modern operating systems, where pipx is a replacement for pip.

$ pip install test
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
    python-xyz', where xyz is the package you are trying to
    install.

I have tried clearing the cache and still had the same issue using alltalk.

A terminal recording of the entire installation process, including the issue at the end, is available here: https://asciinema.org/a/R321XEnayqKO2tDNfDSS3fAoG

I am currently testing the beta branch, and will see how it goes.

erew123 commented 3 months ago

Thanks for that! Makes it much easier to see whats going on.

Ok my suggestion is this (with v1 alltalk):

start_environment.sh and get into the Python environment:
pip cache purge
pip uninstall torch
pip uninstall torchaudio
pip install torch==2.2.1+cu121 torchaudio==2.2.1+cu121 --upgrade --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu121

That should downgrade the torch version correctly. Looks like somewhere a > had crept into the atsetup.sh so that will have been what resulted in pulling the 2.3.1 version.

After that, you should be able to pip show torch and you should have 2.2.1 and things should be ok at that point, though you may have to pip uninstall deepspeed then re-install the wheel file.

ObscenityIB commented 3 months ago

Happy to report that the AllTalk BETA worked with no issues :)

Afterwards, v1 started working as well.

Thanks for your help on this issue. :+1:

erew123 / alltalk_tts

Expected String and got Path #251