search

Artrajz / vits-simple-api

A simple VITS HTTP API, developed by extending Moegoe with additional features.
GNU Affero General Public License v3.0
777 stars 116 forks source link

请问是否支持Bert-VITS2-Extra中文特化版本 #134

Closed DoshideDK closed 7 months ago

DoshideDK commented 7 months ago

运行环境

问题复现步骤

Artrajz commented 7 months ago

支持中文特化版，当前版本的特化版存在bug，请暂时切换至v0.6.5版本使用

DoshideDK commented 7 months ago

支持中文特化版，当前版本的特化版存在bug，请暂时切换至v0.6.5版本使用

测试了一下v0.65，电流音没了，但输出仍然失真

DoshideDK commented 7 months ago

应该没有报错信息 ` INFO:root:Loading yaml from D:\Program Files\vits-simple-api-v0.6.5\config.yml Building prefix dict from the default dictionary ... DEBUG:jieba:Building prefix dict from the default dictionary ... Loading model from cache C:\Users\dkh\AppData\Local\Temp\jieba.cache DEBUG:jieba:Loading model from cache C:\Users\dkh\AppData\Local\Temp\jieba.cache Loading model cost 0.289 seconds. DEBUG:jieba:Loading model cost 0.289 seconds. Prefix dict has been built successfully. DEBUG:jieba:Prefix dict has been built successfully. 2024-01-24 14:16:32 [INFO] [model_handler.load_bert:134] Loading BERT model: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v2-large-japanese-char-wwm 2024-01-24 14:16:33 [INFO] [model_handler.load_bert:143] Success loading: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v2-large-japanese-char-wwm 2024-01-24 14:16:33 [INFO] [model_handler.load_bert:134] Loading BERT model: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v3-large D:\Program Files\vits-simple-api-v0.6.5\py310\lib\site-packages\transformers\convert_slow_tokenizer.py:473: UserWarning: The sentencepiece tokenizer that you are converting to a fast tokenizer uses the byte fallback option which is not implemented in the fast tokenizers. In practice this means that the fast version of the tokenizer can produce unknown tokens whereas the sentencepiece version would have converted these unknown tokens into a sequence of byte tokens matching the original piece of text. warnings.warn( Some weights of DebertaV2ForMaskedLM were not initialized from the model checkpoint at D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v3-large and are newly initialized: ['cls.predictions.decoder.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. 2024-01-24 14:16:34 [INFO] [model_handler.load_bert:143] Success loading: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v3-large 2024-01-24 14:16:34 [INFO] [model_handler.load_bert:134] Loading BERT model: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/chinese-roberta-wwm-ext-large Some weights of the model checkpoint at D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/chinese-roberta-wwm-ext-large were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'cls.seq_relationship.weight', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias']

gushuaialan1 commented 7 months ago

应该没有报错信息 ` INFO:root:Loading yaml from D:\Program Files\vits-simple-api-v0.6.5\config.yml Building prefix dict from the default dictionary ... DEBUG:jieba:Building prefix dict from the default dictionary ... Loading model from cache C:\Users\dkh\AppData\Local\Temp\jieba.cache DEBUG:jieba:Loading model from cache C:\Users\dkh\AppData\Local\Temp\jieba.cache Loading model cost 0.289 seconds. DEBUG:jieba:Loading model cost 0.289 seconds. Prefix dict has been built successfully. DEBUG:jieba:Prefix dict has been built successfully. 2024-01-24 14:16:32 [INFO] [model_handler.load_bert:134] Loading BERT model: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v2-large-japanese-char-wwm 2024-01-24 14:16:33 [INFO] [model_handler.load_bert:143] Success loading: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v2-large-japanese-char-wwm 2024-01-24 14:16:33 [INFO] [model_handler.load_bert:134] Loading BERT model: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v3-large D:\Program Files\vits-simple-api-v0.6.5\py310\lib\site-packages\transformers\convert_slow_tokenizer.py:473: UserWarning: The sentencepiece tokenizer that you are converting to a fast tokenizer uses the byte fallback option which is not implemented in the fast tokenizers. In practice this means that the fast version of the tokenizer can produce unknown tokens whereas the sentencepiece version would have converted these unknown tokens into a sequence of byte tokens matching the original piece of text. warnings.warn( Some weights of DebertaV2ForMaskedLM were not initialized from the model checkpoint at D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v3-large and are newly initialized: ['cls.predictions.decoder.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. 2024-01-24 14:16:34 [INFO] [model_handler.load_bert:143] Success loading: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/deberta-v3-large 2024-01-24 14:16:34 [INFO] [model_handler.load_bert:134] Loading BERT model: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/chinese-roberta-wwm-ext-large Some weights of the model checkpoint at D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/chinese-roberta-wwm-ext-large were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'cls.seq_relationship.weight', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias']

  • This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).

  • This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). 2024-01-24 14:16:35 [INFO] [model_handler.load_bert:143] Success loading: D:\Program Files\vits-simple-api-v0.6.5\bert_vits2/bert/chinese-roberta-wwm-ext-large 2024-01-24 14:16:36 [INFO] [utils.load_checkpoint:65] Loaded checkpoint 'D:\Program Files\vits-simple-api-v0.6.5\Model\Yennefer\G_150.pth' (iteration 38) 2024-01-24 14:16:36 [INFO] [ModelManager._load_model_from_path:234] model_type:BERT-VITS2 model_id:0 n_speakers:1 model_path:D:\Program Files\vits-simple-api-v0.6.5\Model\Yennefer\G_150.pth 2024-01-24 14:16:36 [INFO] [ModelManager.log_device_info:148] PyTorch Version: 1.13.1+cu117 Cuda available:True Device type:cuda 2024-01-24 14:16:36 [INFO] [ModelManager.log_device_info:153] Using GPU on NVIDIA GeForce RTX 4090, GPU Device Index: None 2024-01-24 14:16:36 [INFO] [ModelManager.model_init:99] [BERT-VITS2] 1 speakers 2024-01-24 14:16:36 [INFO] [ModelManager.model_init:101] 1 speakers in total. 2024-01-24 14:16:36 [INFO] [phrases_dict.phrases_dict_init:33] Loading phrases_dict 2024-01-24 14:16:36 [DEBUG] [phrases_dict.load_phrases_from_file:27] File D:\Program Files\vits-simple-api-v0.6.5/phrases_dict.txt not found. You can create D:\Program Files\vits-simple-api-v0.6.5/phrases_dict.txt and write your phrases_dict. 2024-01-24 14:16:37 [DEBUG] [win32._get_localzone_name:58] Looking up time zone info from registry 2024-01-24 14:16:37 [INFO] [base.start:181] Scheduler started 2024-01-24 14:16:37 [INFO] [base._real_add_job:895] Added job "clean_task" to job store "default"

  • Serving Flask app 'app'

  • Debug mode: off 2024-01-24 14:16:37 [INFO] [_internal._log:187] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

  • Running on all addresses (0.0.0.0)

  • Running on http://127.0.0.1:23456

  • Running on http://192.168.31.163:23456 2024-01-24 14:16:37 [INFO] [_internal._log:187] Press CTRL+C to quit `

应该是你的config.json中的vision没有正确修改

Artrajz commented 7 months ago

检查一下config.json是否改成了特化版的版本号extra或者zh-clap