我无法使用2.1版本的vits2模型

ArackLiceve commented 9 months ago

运行环境

操作系统 (Linux/macOS/Windows)：Windows
部署方式 (Docker/windows快速部署包/自己搭的环境)：windows快速部署包
Python 版本 (如果是部署包可不填)：
代码版本/部署包版本: v0.6.0

问题描述

使用v2.1 emo来训练了一个模型，导入后无法使用
使用autodl平台进行模型训练
配置文件config.json内的版本为"version": "2.1"
不知道是不是软件还没适配v2.1或者是我个人原因造成的，如果是我个人原因造成的请帮帮我，我会尽可能提供更多的资料。

Artrajz commented 9 months ago

可以看一下日志中模型加载错误的报错吗？

ArackLiceve commented 9 months ago

你需要的因该是这个模型加载报错的日志

2023-12-03 22:42:32 [INFO] [_internal._log:187] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

Running on all addresses (0.0.0.0)
Running on http://127.0.0.1:23456
Running on http://192.168.42.3:23456 2023-12-03 22:42:32 [INFO] [_internal._log:187] Press CTRL+C to quit 2023-12-03 22:44:43 [INFO] [views.load_model:49] Loading model model_path: huohuo/G_10000.pth config_path: huohuo/config.json 2023-12-03 22:44:43 [INFO] [bert_handler.load_bert:93] Loading BERT model: D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/bert/deberta-v3-large 2023-12-03 22:44:43 [ERROR] [bert_handler.load_bert:101] Failed loading D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/bert/deberta-v3-large. Unable to load vocabulary from file. Please check that the provided vocabulary is accessible and not corrupted. 2023-12-03 22:44:43 [INFO] [bert_handler.load_bert:102] Trying to download. 2023-12-03 22:44:44 [INFO] [bert_handler._download_model:86] File already exists and verified successfully! 2023-12-03 22:44:44 [INFO] [bert_handler.load_bert:93] Loading BERT model: D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/bert/deberta-v3-large 2023-12-03 22:44:44 [ERROR] [bert_handler.load_bert:101] Failed loading D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/bert/deberta-v3-large. Unable to load vocabulary from file. Please check that the provided vocabulary is accessible and not corrupted. 2023-12-03 22:44:44 [INFO] [bert_handler.load_bert:102] Trying to download. 2023-12-03 22:44:45 [INFO] [bert_handler._download_model:86] File already exists and verified successfully! 2023-12-03 22:44:45 [INFO] [bert_handler.load_bert:93] Loading BERT model: D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/bert/deberta-v3-large 2023-12-03 22:44:45 [ERROR] [bert_handler.load_bert:101] Failed loading D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/bert/deberta-v3-large. Unable to load vocabulary from file. Please check that the provided vocabulary is accessible and not corrupted. 2023-12-03 22:44:45 [INFO] [bert_handler.load_bert:102] Trying to download. 2023-12-03 22:44:45 [INFO] [bert_handler._download_model:86] File already exists and verified successfully! 2023-12-03 22:44:45 [ERROR] [bert_handler.load_bert:109] Failed to load D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/bert/deberta-v3-large after 3 retries. 2023-12-03 22:44:45 [INFO] [ModelManager.load_model:266] Loading failed. Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim. 2023-12-03 22:44:45 [ERROR] [ModelManager.load_model:267] Traceback (most recent call last): File "D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\ModelManager.py", line 253, in load_model model_data = self._load_model_from_path(model_path, config_path) File "D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\ModelManager.py", line 203, in _load_model_from_path self.load_emotion_model(WAV2VEC2_LARGE_ROBUST_12_FT_EMOTION_MSP_DIM) File "D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\ModelManager.py", line 355, in load_emotion_model self.emotion_model = EmotionModel.from_pretrained(model_path).to(self.device) File "D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\transformers\modeling_utils.py", line 2992, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory D:\桌面文件\下载列表\model\vits-simple-api-windows-gpu-v0.6.0\bert_vits2/emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim.

2023-12-03 22:44:45 [INFO] [_internal._log:187] 192.168.42.3 - - [03/Dec/2023 22:44:45] "POST /admin/load_model HTTP/1.1" 500 -

Artrajz commented 9 months ago

日志中说缺少wav2vec2-large-robust-12-ft-emotion-msp-dim模型，你可以在这里下载下载pytorch_model.bin后放入bert_vits2/emotional/wav2vec2-large-robust-12-ft-emotion-msp-dim文件夹中

Ikaros-521 commented 9 months ago

借楼，首次加载v2.1的模型，报错

File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\bert_vits2\get_emo.py", line 83, in get_emo
    wav, sr = librosa.load(audio, 16000)
TypeError: load() takes 1 positional argument but 2 were given

修改后bert_vits2\get_emo.py正常运行

wav, sr = librosa.load(audio, sr=16000)

加载音频第一次成功，后面第二次推理时报错，加载失败

2023-12-07 00:19:13 [ERROR] [app.log_exception:1744] Exception on /voice/bert-vits2 [POST]
Traceback (most recent call last):
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\flask\app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\flask\app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\flask\app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\flask\app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\tts_app\voice_api\auth.py", line 9, in check_api_key
    return func(*args, **kwargs)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\tts_app\voice_api\views.py", line 480, in voice_bert_vits2_api
    audio = tts_manager.bert_vits2_infer(state)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\TTSManager.py", line 375, in bert_vits2_infer
    audio = model.infer(sentence, state["id"], lang, state["sdp_ratio"], state["noise"],
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\bert_vits2\bert_vits2.py", line 171, in infer
    emo = self.get_emo_(reference_audio, emotion).to(self.device).unsqueeze(0)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\bert_vits2\bert_vits2.py", line 161, in get_emo_
    get_emo(reference_audio, self.emotion_model, self.processor)) if reference_audio else torch.Tensor(
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\bert_vits2\get_emo.py", line 84, in get_emo
    wav, sr = librosa.load(audio, sr=16000)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\librosa\core\audio.py", line 185, in load
    raise exc
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\librosa\core\audio.py", line 175, in load
    y, sr_native = __soundfile_load(path, offset, duration, dtype)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\librosa\core\audio.py", line 208, in __soundfile_load
    context = sf.SoundFile(path)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\soundfile.py", line 658, in __init__
    self._file = self._open(file, mode_int, closefd)
  File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\py310\lib\site-packages\soundfile.py", line 1216, in _open
    raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
soundfile.LibsndfileError: Error opening <FileStorage: '1.wav' ('audio/wav')>: Format not recognised.
2023-12-07 00:19:13 [INFO] [_internal._log:187] 127.0.0.1 - - [07/Dec/2023 00:19:13] "POST /voice/bert-vits2 HTTP/1.1" 500 -

Artrajz commented 9 months ago

File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\bert_vits2\get_emo.py", line 83, in get_emo wav, sr = librosa.load(audio, 16000) TypeError: load() takes 1 positional argument but 2 were given

应该是librosa的版本问题，requirements.txt里没有指定版本，所以部署包里自动下载了最新版本。部署包里更换版本：

py310\python.exe -m pip install librosa==0.9.1

soundfile.LibsndfileError: Error opening <FileStorage: '1.wav' ('audio/wav')>: Format not recognised.

不清楚是什么问题，我这里不管是第几次推理，加载上传的音频时都是正常的。可能也是librosa的版本问题。

Ikaros-521 commented 9 months ago

File "D:\aizb\vits-simple-api-windows-gpu-v0.6.0\bert_vits2\get_emo.py", line 83, in get_emo wav, sr = librosa.load(audio, 16000) TypeError: load() takes 1 positional argument but 2 were given

应该是librosa的版本问题，requirements.txt里没有指定版本，所以部署包里自动下载了最新版本。部署包里更换版本：
py310\python.exe -m pip install librosa==0.9.1
soundfile.LibsndfileError: Error opening <FileStorage: '1.wav' ('audio/wav')>: Format not recognised.

不清楚是什么问题，我这里不管是第几次推理，加载上传的音频时都是正常的。可能也是librosa的版本问题。

ok 值得一试🥰

Artrajz / vits-simple-api

我无法使用2.1版本的vits2模型 #111

运行环境

问题描述