mudler / LocalAI

:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
https://localai.io
MIT License
21.88k stars 1.68k forks source link

Can not load tts_models/multilingual TTS models #2327

Open edisonzf2020 opened 2 months ago

edisonzf2020 commented 2 months ago

LocalAI version: 2.15.0

Docker: image: quay.io/go-skynet/local-ai:master-cublas-cuda12-ffmpeg Environment, CPU architecture, OS, and Version:

Ubuntu 22.04 GPU Describe the bug

test 3 models : 1: tts_models/multilingual/multi-dataset/xtts_v2 2: tts_models/multilingual/multi-dataset/xtts_v1.1 4: tts_models/multilingual/multi-dataset/bark

curl -L http://192.168.1.19:8090/tts \
    -H "Content-Type: application/json" \
    -d '{
"backend": "coqui",
"input": "Bonjour Madame ! Comment allez-vous ?",
"model": "tts_models/multilingual/multi-dataset/xtts_v2",
"voice": "Ana Florence"
}' | aplay -D pipewire -

To Reproduce

8:25AM ERR Server error error="could not load model (no success): Unexpected err=UnpicklingError(\"invalid load key, '<'.\"), type(err)=<class '_pickle.UnpicklingError'>" ip=172.22.0.1 latency=13.523714559s method=POST status=500 url=/v1/audio/speech
8:25AM INF Loading model 'tts_models/multilingual/multi-dataset/bark' with backend coqui
8:26AM INF Loading model 'tts_models/multilingual/multi-dataset/xtts_v2' with backend coqui
8:26AM ERR Server error error="could not load model (no success): Unexpected err=UnpicklingError(\"invalid load key, '<'.\"), type(err)=<class '_pickle.UnpicklingError'>" ip=172.22.0.1 latency=13.108962696s method=POST status=500 url=/v1/audio/speech
8:26AM INF Loading model 'tts_models/multilingual/multi-dataset/bark' with backend coqui
8:26AM ERR Server error error="could not load model (no success): Unexpected err=RuntimeError('input(): lost sys.stdin'), type(err)=<class 'RuntimeError'>" ip=192.168.1.28 latency=6.335075199s method=POST status=500 url=/tts

Expected behavior

Logs

Additional context

blob42 commented 1 month ago

Hi. This will be solved when #2308 is merged.