sunbin728
commented
8 months ago 我发现是GPT_SoVITS/AR/models/t2s_model.py文件的infer_panel方法中，过早的samples[0, 0] == self.EOS，不知道是什么原因。
if torch.argmax(logits, dim=-1)[0] == self.EOS or samples[0, 0] == self.EOS:
# print(torch.argmax(logits, dim=-1)[0] == self.EOS, samples[0, 0] == self.EOS)
stop = True
if stop:
# if prompts.shape[1] == y.shape[1]:
# y = torch.concat([y, torch.zeros_like(samples)], dim=1)
# print("bad zero prediction")
if y.shape[1]==0:
y = torch.concat([y, torch.zeros_like(samples)], dim=1)
print("bad zero prediction")
print(f"T2S Decoding EOS [{prefix_len} -> {y.shape[1]}]")
break下面是我添加的打印： 2024-03-07 17:08:45.431 | DEBUG | AR.models.t2s_model:infer_panel:408 - idx=0, logits=tensor([[ -5.4335, -9.7142, -7.5205, ..., -9.5499, -12.1951, -8.4418]], device='cuda:0'), samples=tensor([[536]], device='cuda:0', dtype=torch.int32) 2024-03-07 17:08:45.443 | DEBUG | AR.models.t2s_model:infer_panel:408 - idx=1, logits=tensor([[-2.6589, -3.9608, -5.4140, ..., -1.9371, -6.5594, 18.1556]], device='cuda:0'), samples=tensor([[1024]], device='cuda:0', dtype=torch.int32) T2S Decoding EOS [90 -> 92] 0%|â | 1/1500 [00:00<00:40, 36.70it/s]
KamioRinn
k18coldwl
linrb685
BankNatchapol
curl --location 'http://127.0.0.1:9880' \ --header 'Content-Type: application/json' \ --data '{ "refer_wav_path":"output/xiaojun/1_0.wav", "prompt_text":"那为什么还有那么多商业领袖？对如此重要的用户体验问题视而不见呢？", "prompt_language": "zh", "text": "今天天气怎么样？", "text_language": "zh" }'
api.py中def get_tts_wav方法的代码： pred_semantic, idx = t2s_model.model.infer_panel( all_phoneme_ids, all_phoneme_len, prompt, bert,
prompt_phone_len=ph_offset,
添加的测试打印
logger.debug(f"pred_semantic.shape={pred_semantic.shape}, idx={idx}")
情况说明： 使用自己微调训练的模型推理时，当text相对比较短时，经常出现idx很小和idx=0的情况，导致生成的语音要么没有声音，要么变成了参考音频。
情况1打印: 2024-03-07 16:20:53.299 | DEBUG | main:get_tts_wav:466 - pred_semantic.shape=torch.Size([1, 205]), idx=9 情况2打印： 2024-03-07 16:26:32.582 | DEBUG | main:get_tts_wav:466 - pred_semantic.shape=torch.Size([1, 196]), idx=0 idx=0时，返回了参考音频。