rsxdalv / tts-generation-webui

TTS Generation Web UI (Bark, MusicGen + AudioGen, Tortoise, RVC, Vocos, Demucs, SeamlessM4T, MAGNet, StyleTTS2, MMS)
https://rsxdalv.github.io/tts-generation-webui/
MIT License
1.51k stars 160 forks source link

Missing core packages (dotenv), the app has not been installed successfully. #335

Closed Xeraster closed 1 week ago

Xeraster commented 2 weeks ago

I'm trying to install this via the installation script. The docker install is also broken for me. But anyway, when I run ./start_tts_webui.sh it does its thing for a bit and gives me the following error:

Finished init app.

Missing core packages (dotenv), the app has not been installed successfully.
Failed to start the app, exiting...

If I try to do docker, there are 2 issues. First, all dns on the entire computer stops working unless I run docker commands with the --network="host" option. If I do that, it gets to this point:

File "/venv/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 179, in resolve
    self.factory.preparer.prepare_linked_requirements_more(reqs)
  File "/venv/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 552, in prepare_linked_requirements_more
    self._complete_partial_requirements(
  File "/venv/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 467, in _complete_partial_requirements
    for link, (filepath, _) in batch_download:
  File "/venv/lib/python3.10/site-packages/pip/_internal/network/download.py", line 183, in __call__
    for chunk in chunks:
  File "/venv/lib/python3.10/site-packages/pip/_internal/cli/progress_bars.py", line 53, in _rich_progress_bar
    for chunk in iterable:
  File "/venv/lib/python3.10/site-packages/pip/_internal/network/utils.py", line 63, in response_chunks
    for chunk in response.raw.stream(
  File "/venv/lib/python3.10/site-packages/pip/_vendor/urllib3/response.py", line 622, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/venv/lib/python3.10/site-packages/pip/_vendor/urllib3/response.py", line 560, in read
    with self._error_catcher():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/venv/lib/python3.10/site-packages/pip/_vendor/urllib3/response.py", line 443, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.")
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

[notice] A new release of pip is available: 23.3.2 -> 24.1.2
[notice] To update, run: pip install --upgrade pip
The command '/bin/sh -c pip3 install --no-cache-dir --upgrade pip==23.3.2 setuptools &&     pip3 install --no-cache-dir torch==$TORCH_VERSION torchvision torchaudio' returned a non-zero code: 2

so docker doesn't work on my system. The installation script doesn't work on my system. I'm not touching that manual installation until I've completely exhausted the other 2 possible methods of installing this. Which of these issues are more likely to be fixable if I just really really dig? The docker installation or the install script installation? I'm on Debian 12.

I think this might be the furthest I've ever got in trying to install any of the github ai voice projects but lemme know if there's an even easier one somewhere. iirc it gets to step 23/24 but i've tried so many different ai text to speech things trying to find one that will work on my computer that its hard to keep track.

Update: the manual installation basically doesn't get anywhere before running into issues. This is what happens when trying to run conda install -y -c conda-forge git nodejs -c pytorch ffmpeg:

## Package Plan ##

  environment location: /home/scott/miniconda3/envs/venv

  added / updated specs:
    - python=3.10

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    pip-24.0                   |  py310h06a4308_0         2.7 MB
    python-3.10.14             |       h955ad1f_1        26.8 MB
    setuptools-69.5.1          |  py310h06a4308_0        1012 KB
    wheel-0.43.0               |  py310h06a4308_0         110 KB
    ------------------------------------------------------------
                                           Total:        30.6 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu 
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6 
  ca-certificates    pkgs/main/linux-64::ca-certificates-2024.3.11-h06a4308_0 
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_1 
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 
  libuuid            pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0 
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 
  openssl            pkgs/main/linux-64::openssl-3.0.14-h5eee18b_0 
  pip                pkgs/main/linux-64::pip-24.0-py310h06a4308_0 
  python             pkgs/main/linux-64::python-3.10.14-h955ad1f_1 
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0 
  setuptools         pkgs/main/linux-64::setuptools-69.5.1-py310h06a4308_0 
  sqlite             pkgs/main/linux-64::sqlite-3.45.3-h5eee18b_0 
  tk                 pkgs/main/linux-64::tk-8.6.14-h39e8969_0 
  tzdata             pkgs/main/noarch::tzdata-2024a-h04d1e81_0 
  wheel              pkgs/main/linux-64::wheel-0.43.0-py310h06a4308_0 
  xz                 pkgs/main/linux-64::xz-5.4.6-h5eee18b_1 
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1 

Proceed ([y]/n)? y

Downloading and Extracting Packages:

Preparing transaction: done                                                                                                                                                                                                                  
Verifying transaction: done                                                                                                                                                                                                                  
Executing transaction: done                                                                                                                                                                                                                  
#
# To activate this environment, use
#
#     $ conda activate venv
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) scott@CrapWagon:/media/hdd/documents_hdd/tts-generation-webui$ conda install -y -c conda-forge git nodejs -c pytorch ffmpeg
/home/scott/miniconda3/lib/python3.12/site-packages/requests/__init__.py:86: RequestsDependencyWarning: Unable to find acceptable character detection dependency (chardet or charset_normalizer).
  warnings.warn(
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: unrecognized arguments: ffmpeg

Edit: I discovered a log file that might be useful: https://pastebin.com/fDmrHu6a unfortunately its actually finding and installing dotenv so whatever hack it takes to fix this is probably going to be pretty hard to figure out.

I already tried the obvious thing by commenting out the dotenv check in.. whatever src file grep says its in. I get this error:

Finished init app.

Traceback (most recent call last):
  File "/media/hdd/documents_hdd/tts-generation-webui/tts-generation-webui/server.py", line 3, in <module>
    import src.utils.dotenv_init as dotenv_init
  File "/media/hdd/documents_hdd/tts-generation-webui/tts-generation-webui/src/utils/dotenv_init.py", line 1, in <module>
    from dotenv import load_dotenv
ModuleNotFoundError: No module named 'dotenv'
Failed to start the app, exiting...

I don't know enough about how pip or conda works to try to devise a way to jury-rig it to have access to a standalone dotenv python package.. or something. Maybe i'll find a way.

I already tried conda install -p tts-generation-webui/installer_files/env dotenv I forgot to save the output but there weren't any outstanding errors. It still entirely ignores the existence of dotenv and I still get the same error about missing dotenv when I try to run sh start_tts_webui.sh. I still don't have anything titles "dotenv" in tts-generation-webui/installer_files/env/pkgs so that manual installation command I came up with did NOTHING. Maybe next i'll try shoehorning some dotenv files in there manually.....

rsxdalv commented 1 week ago

Great job reporting issues and my apologies for the bugs that you have uncovered!

The installer failed because in yesterday's update I left a --dry-run in the install command because I was debugging installation problems, so unfortunately it was just not installing any of the packages. I am assuming you did not try the installer prior to trying the docker, right?

This helped me debug another issue in the manual instructions, thank you:

conda: error: unrecognized arguments: ffmpeg
rsxdalv commented 1 week ago

As for the docker - would it be possible to use the pre-built docker images?

I have updated the docker setup section in README:

instead of docker build -t rsxdalv/tts-generation-webui . doing docker pull ghcr.io/rsxdalv/tts-generation-webui:main

wazzzy commented 1 week ago

Any updates on this issue further?

rsxdalv commented 1 week ago

Hi, @wazzzy which problem are you seeing? Is it the missing dotenv? I have made several patches to fix the installer, so please let me know what errors/clues you are seeing.

wazzzy commented 1 week ago

Hello @rsxdalv I get this error

Already up to date.
No updates found, skipping...
Missing core packages (dotenv), the app has not been installed successfully.
Failed to start the app, exiting...
rsxdalv commented 1 week ago

Can you delete the .git folder and try running the start script again, please?

Also, if you would like to chat directly you can join the discord. https://discord.gg/3JbBrKrH

wazzzy commented 1 week ago

Delete git files worked but now facing another issue

OSError: dlopen(/tts-generation-webui/installer_files/env/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so, 0x0006): Symbol not found: __ZN2at4_ops10empty_like4callERKNS_6TensorENSt3__18optionalIN3c1010ScalarTypeEEENS6_INS7_6LayoutEEENS6_INS7_6DeviceEEENS6_IbEENS6_INS7_12MemoryFormatEEE
  Referenced from: <FCB6BB4B-958C-3899-BD17-48263A6037B9> /tts-generation-webui/installer_files/env/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so
  Expected in:     <78DA4C67-8BC7-3995-8324-D2FD95E57BA4> /tts-generation-webui/installer_files/env/lib/python3.10/site-packages/torch/lib/libtorch_cpu.dylib
Failed to start the app, exiting...
rsxdalv commented 1 week ago

That appears to be a corruption of the torch installation. There should be an output.log in the installer files directory which could show the reason why.

It can be fixed by reinstalling torch. If you are using cpu, it would be this:

export MICROMAMBA_EXE="./installer_files/mamba/micromamba"

$MICROMAMBA_EXE run -p ./installer_files/env bash

conda install -y -k pytorch torchvision torchaudio cpuonly ffmpeg -c pytorch

For CUDA it is this:

conda install -y -k pytorch[version=2,build=py3.10_cuda11.7*] torchvision torchaudio pytorch-cuda=11.7 cuda-toolkit ninja ffmpeg -c pytorch -c nvidia/label/cuda-11.7.0 -c nvidia

it might require a --force-reinstall flag if it refuses to install them.

conda install -y -k --force-reinstall pytorch torchvision torchaudio cpuonly ffmpeg -c pytorch

or

conda install -y -k --force-reinstall pytorch[version=2,build=py3.10_cuda11.7*] torchvision torchaudio pytorch-cuda=11.7 cuda-toolkit ninja ffmpeg -c pytorch -c nvidia/label/cuda-11.7.0 -c nvidia
rsxdalv commented 1 week ago

If possible, please provide the ./installer_scripts/output.log file, I have a few ideas why this error happened but if I can confirm them I can fix it for all users.

c45671234 commented 1 week ago

Here's the contents of my log file. I'm getting the following error message.

"Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'torch' Missing core packages (dotenv), the app has not been installed successfully. Failed to start the app, exiting..."

git fetch From https://github.com/rsxdalv/tts-generation-webui

  • [new branch] gh-pages -> origin/gh-pages
  • [new branch] main -> origin/main
  • [new branch] new-model-demos -> origin/new-model-demos git reset --hard origin/main HEAD is now at 566f811 make installer auto-update as it runs git branch --set-upstream-to=origin/main Branch 'main' set up to track remote branch 'main' from 'origin'. Checking if Torch is installed... python -c import importlib.util; import sys; package_name = "torch"; spec = importlib.util.find_spec(package_name); sys.exit(0) if spec else sys.exit(1) Torch is not installed. Starting installation...

Using saved GPU choice: NVIDIA GPU Error during installation: $ is not defined Installing node_modules...

cd react-ui && npm install

removed 1 package, and audited 481 packages in 1s

146 packages are looking for funding run npm fund for details

8 vulnerabilities (1 low, 4 moderate, 3 high)

To address issues that do not require attention, run: npm audit fix

To address all issues, run: npm audit fix --force

Run npm audit for details. Successfully installed node_modules Building react-ui...

cd react-ui && npm run build

tts-generation-webui-react@0.1.0 build next build

Route (app) Size First Load JS ─ λ /api/upload 0 B 0 B

Route (pages) Size First Load JS ┌ ● / 1.55 kB 106 kB ├ /_app 0 B 92.4 kB ├ ○ /404 178 B 92.6 kB ├ λ /api/gradio/[name] 0 B 92.4 kB ├ λ /api/webui-generations/[...name] 0 B 92.4 kB ├ ○ /bark 3.8 kB 119 kB ├ ○ /bark/bark_settings 2.07 kB 106 kB ├ ○ /bark/bark_voice_generation 2.95 kB 114 kB ├ ○ /bark/vocos_npz 2.7 kB 114 kB ├ ○ /bark/vocos_wav 2.88 kB 114 kB ├ ● /bark/voices 539 B 105 kB ├ ○ /demucs 3.08 kB 115 kB ├ ● /generations 188 B 104 kB ├ ○ /gpu_info 965 B 105 kB ├ ● /history/[name] 572 B 105 kB ├ ├ /history/outputs ├ └ /history/favorites ├ ○ /magnet 1.5 kB 118 kB ├ ○ /maha-tts 2.3 kB 117 kB ├ ○ /mms 1.5 kB 127 kB ├ ○ /musicgen 2.67 kB 117 kB ├ ○ /pipeline 6.05 kB 165 kB ├ ○ /rvc 4.53 kB 116 kB ├ ○ /tortoise 1.38 kB 143 kB ├ ○ /vallex 2.57 kB 117 kB └ ● /voice-drafts 630 B 105 kB

λ (Server) server-side renders at runtime (uses getInitialProps or getServerSideProps) ○ (Static) automatically rendered as static HTML (uses no initial props) ● (SSG) automatically generated as static HTML + JSON (uses getStaticProps)

Successfully built react-ui Checking if torch has CUDA...

python -c import torch; exit(0 if torch.cuda.is_available() else 1)

Finished init app.

Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'torch' Missing core packages (dotenv), the app has not been installed successfully. Failed to start the app, exiting...

rsxdalv commented 1 week ago

Here's the contents of my log file. I'm getting the following error message.

"Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'torch' Missing core packages (dotenv), the app has not been installed successfully. Failed to start the app, exiting..."

git fetch From https://github.com/rsxdalv/tts-generation-webui

  • [new branch] gh-pages -> origin/gh-pages
  • [new branch] main -> origin/main
  • [new branch] new-model-demos -> origin/new-model-demos

git reset --hard origin/main HEAD is now at 566f811 make installer auto-update as it runs git branch --set-upstream-to=origin/main Branch 'main' set up to track remote branch 'main' from 'origin'. Checking if Torch is installed... python -c import importlib.util; import sys; package_name = "torch"; spec = importlib.util.find_spec(package_name); sys.exit(0) if spec else sys.exit(1) Torch is not installed. Starting installation...

Using saved GPU choice: NVIDIA GPU Error during installation: $ is not defined Installing node_modules...

cd react-ui && npm install

removed 1 package, and audited 481 packages in 1s

146 packages are looking for funding run npm fund for details

8 vulnerabilities (1 low, 4 moderate, 3 high)

To address issues that do not require attention, run: npm audit fix

To address all issues, run: npm audit fix --force

Run npm audit for details. Successfully installed node_modules Building react-ui...

cd react-ui && npm run build

tts-generation-webui-react@0.1.0 build next build

  • info Creating an optimized production build...
  • info Compiled successfully
  • info Linting and checking validity of types...
  • info Collecting page data...
  • info Generating static pages (0/24)
  • info Generating static pages (6/24)
  • info Generating static pages (12/24)
  • info Generating static pages (18/24)
  • info Generating static pages (24/24)
  • info Finalizing page optimization...

Route (app) Size First Load JS ─ λ /api/upload 0 B 0 B

  • First Load JS shared by all 0 B

Route (pages) Size First Load JS ┌ ● / 1.55 kB 106 kB ├ /_app 0 B 92.4 kB ├ ○ /404 178 B 92.6 kB ├ λ /api/gradio/[name] 0 B 92.4 kB ├ λ /api/webui-generations/[...name] 0 B 92.4 kB ├ ○ /bark 3.8 kB 119 kB ├ ○ /bark/bark_settings 2.07 kB 106 kB ├ ○ /bark/bark_voice_generation 2.95 kB 114 kB ├ ○ /bark/vocos_npz 2.7 kB 114 kB ├ ○ /bark/vocos_wav 2.88 kB 114 kB ├ ● /bark/voices 539 B 105 kB ├ ○ /demucs 3.08 kB 115 kB ├ ● /generations 188 B 104 kB ├ ○ /gpu_info 965 B 105 kB ├ ● /history/[name] 572 B 105 kB ├ ├ /history/outputs ├ └ /history/favorites ├ ○ /magnet 1.5 kB 118 kB ├ ○ /maha-tts 2.3 kB 117 kB ├ ○ /mms 1.5 kB 127 kB ├ ○ /musicgen 2.67 kB 117 kB ├ ○ /pipeline 6.05 kB 165 kB ├ ○ /rvc 4.53 kB 116 kB ├ ○ /tortoise 1.38 kB 143 kB ├ ○ /vallex 2.57 kB 117 kB └ ● /voice-drafts 630 B 105 kB

  • First Load JS shared by all 95.8 kB ├ chunks/main-f2f37a715f8b92e3.js 83.9 kB ├ chunks/pages/_app-503f1e0a2ea8898e.js 6.53 kB ├ chunks/webpack-09f7c4a233429073.js 1.98 kB └ css/c88ad04737390e95.css 3.38 kB

λ (Server) server-side renders at runtime (uses getInitialProps or getServerSideProps) ○ (Static) automatically rendered as static HTML (uses no initial props) ● (SSG) automatically generated as static HTML + JSON (uses getStaticProps)

Successfully built react-ui Checking if torch has CUDA...

python -c import torch; exit(0 if torch.cuda.is_available() else 1)

Finished init app.

Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'torch' Missing core packages (dotenv), the app has not been installed successfully. Failed to start the app, exiting...

thank you, fixed that just now

c45671234 commented 1 week ago

@rsxdalv Thank you. The latest update fixed the install issue. I appreciate the quick fix.

rsxdalv commented 1 week ago

I have added several fixes to the installer and performed multiple tests so I believe this has been resolved. If the issue persists please let me know or reopen the issue.