Ikaros-521 / AI-Vtuber

AI Vtuber是一个由 【ChatterBot/ChatGPT/claude/langchain/chatglm/text-gen-webui/闻达/千问/kimi/ollama】 驱动的虚拟主播【Live2D/UE/xuniren】,可以在 【Bilibili/抖音/快手/微信视频号/拼多多/斗鱼/YouTube/twitch/TikTok】 直播中与观众实时互动 或 直接在本地进行聊天。它使用TTS技术【edge-tts/VITS/elevenlabs/bark/bert-vits2/睿声】生成回答并可以选择【so-vits-svc/DDSP-SVC】变声;指令协同SD画图。
https://ikaros-521.github.io/Luna-Docs/site/
GNU General Public License v3.0
2.8k stars 430 forks source link

🐛 一些问题。。。 | [Bug] Some problem... #809

Closed cracya01 closed 4 months ago

cracya01 commented 4 months ago

您使用的版本? | Your usage version?

20240417

您使用的python版本&系统环境? | Your usage python version & system?

python:3.10.11 系统:win11

您使用的场景? | Your usage scenarios?

faster_whisper,qwen,voicemeeter,obs,vtuber studio,bert-vits2

您做了什么操作? | What did you do?

在AI Vtuber声卡输入设备选择了voicemeeter的虚拟麦之后,出现了这种情况,切换回来物理麦也没用。

您遇到了什么问题? | What are your problems?

使用voicemeeter的虚拟麦作为声卡输入,一按Q就会出现以下内容,没有任何停顿等待直接出现了识别结果,虚拟麦输入输出都正常,每次都有这个R.I.P.

{17B97F0B-DB64-4169-9B0D-9A37D7475822}

2024-05-10 17:49:21 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:433] - INFO: 识别结果:次回予告。 2024-05-10 17:49:21 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': '弹幕信息', 'username': '主人', 'user_face': 'https://robohash.org/ui', 'content_type': 'question', 'content': '次回予告。', 'timestamp': '2024-05-10 17:49:21'}} 2024-05-10 17:49:22 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': 'qwen', 'username': '主人', 'content_type': 'answer', 'content': '好的,我会尽力的。', 'timestamp': '2024-05-10 17:49:22'}} 2024-05-10 17:49:22 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:2116] - INFO: [AI回复主人]:好的,我会尽力的。 py[line:426] - INFO: [0.00s -> 28.16s] R.I.P. 2024-05-10 17:49:34 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:433] - INFO: 识别结果: R.I.P.。 2024-05-10 17:49:34 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': '弹幕信息', 'username': '主人', 'user_face': 'https://robohash.org/ui', 'content_type': 'question', 'content': ' R.I.P.。', 'timestamp': '2024-05-10 17:49:34'}} 2024-05-10 17:49:34 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': 'qwen', 'username': '主人', 'content_type': 'answer', 'content': '很遗憾听到这个消息。', 'timestamp': '2024-05-10 17:49:34'}}
2024-05-10 17:49:34 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:2116] - INFO: [AI回复主人]:很遗憾听到这个消息。 2024-05-10 17:49:34 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 17:49:35 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【很遗憾听到这个消息。】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_5.wav

您期望的结果是怎样的? | What is your expected outcome?

使用虚拟麦作为输入。

cracya01 commented 4 months ago

如果开启连续对话使用虚拟麦作为输入,会一直输出下面内容,而且没有任何停顿,经确认虚拟麦并没有任何声音发出。


2024-05-10 18:05:31 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:484] - INFO: 检测到单击键盘 q,即将开始录音~ 2024-05-10 18:05:32 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:294] - INFO: faster_whisper 模型加载中,请稍后... 2024-05-10 18:05:37 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:300] - INFO: faster_whisper 模型加载完毕,可以开始说话了喵~ 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:262] - INFO: [录入中……] 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:268] - INFO: [语音录入完成] 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:299] - INFO: Processing audio with duration 00:01.216 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:353] - INFO: Detected language 'en' with probability 0.36 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:426] - INFO: [0.00s -> 0.24s] Thank you. 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:433] - INFO: 识别结果: Thank you.。 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:262] - INFO: [录入中……] 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:268] - INFO: [语音录入完成] 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:299] - INFO: Processing audio with duration 00:02.848 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': '弹幕信息', 'username': '主人', 'user_face': 'https://robohash.org/ui', 'content_type': 'question', 'content': ' Thank you.。', 'timestamp': '2024-05-10 18:05:38'}} 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\langid\langid.py[line:162] - INFO: initializing identifier 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:353] - INFO: Detected language 'en' with probability 0.34 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:426] - INFO: [0.00s -> 2.00s] Thanks for watching! 2024-05-10 18:05:38 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:433] - INFO: 识别结果: Thanks for watching!。 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': 'qwen', 'username': '主人', 'content_type': 'answer', 'content': '不客气!有什么我可以帮助你的吗?', 'timestamp': '2024-05-10 18:05:39'}} 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:2116] - INFO: [AI回复主人]:不客气!有什么我可以帮助你的吗? 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:262] - INFO: [录入中……] 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:268] - INFO: [语音录入完成] 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': '弹幕信息', 'username': '主人', 'user_face': 'https://robohash.org/ui', 'content_type': 'question', 'content': ' Thanks for watching!。', 'timestamp': '2024-05-10 18:05:39'}} 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:299] - INFO: Processing audio with duration 00:02.208 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': 'qwen', 'username': '主人', 'content_type': 'answer', 'content': '不用谢,我很高兴能帮上忙!', 'timestamp': '2024-05-10 18:05:39'}}2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:2116] - INFO: [AI回复主人]:不用谢,我很高兴能帮上忙! 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:1795] - INFO: 【触发按键映射】关键词:高兴 按键:['x'] 2024-05-10 18:05:39 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:353] - INFO: Detected language 'ja' with probability 0.30 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:426] - INFO: [0.00s -> 1.22s] ご視聴ありがとうございました 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:433] - INFO: 识别结果:ご視聴ありがとうございました。 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【不客气!有什么我可以帮助你的吗?】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_2.wav 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:262] - INFO: [录入中……] 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:268] - INFO: [语音录入完成] 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': '弹幕信息', 'username': '主人', 'user_face': 'https://robohash.org/ui', 'content_type': 'question', 'content': 'ご視聴ありがとうご ざいました。', 'timestamp': '2024-05-10 18:05:40'}} 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:299] - INFO: Processing audio with duration 00:30.752 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': 'qwen', 'username': '主人', 'content_type': 'answer', 'content': '非常感谢您的观看!', 'timestamp': '2024-05-10 18:05:40'}}
2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:2116] - INFO: [AI回复主人]:非常感谢您的观看! 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【不用谢,我很高兴能帮上忙!】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_3.wav 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:353] - INFO: Detected language 'ja' with probability 0.43 2024-05-10 18:05:40 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【非常感谢您的观看!】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_4.wav 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:426] - INFO: [30.00s -> 54.20s] , , , , , 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:433] - INFO: 识别结果: , , , , ,。 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:262] - INFO: [录入中……] 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:268] - INFO: [语音录入完成] 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': '弹幕信息', 'username': '主人', 'user_face': 'https://robohash.org/ui', 'content_type': 'question', 'content': ' , , , , ,。', 'timestamp': '2024-05-10 18:05:52'}} 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:299] - INFO: Processing audio with duration 00:04.000 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:353] - INFO: Detected language 'nn' with probability 0.42 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': 'qwen', 'username': '主人', 'content_type': 'answer', 'content': '对不起,我无法回答这个问题。您有其他问题需要我的帮助吗?', 'timestamp': '2024-05-10 18:05:52'}} 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:2116] - INFO: [AI回复主人]:对不起,我无法回答这个问题。您有其他问题需要我的帮助吗? 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:426] - INFO: [0.00s -> 3.02s] Tekstet av Nicolai Winther 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:433] - INFO: 识别结果: Tekstet av Nicolai Winther。 2024-05-10 18:05:52 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:262] - INFO: [录入中……] 2024-05-10 18:05:53 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\main.py[line:268] - INFO: [语音录入完成] 2024-05-10 18:05:53 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': '弹幕信息', 'username': '主人', 'user_face': 'https://robohash.org/ui', 'content_type': 'question', 'content': ' Tekstet av Nicolai Winther。', 'timestamp': '2024-05-10 18:05:53'}} 2024-05-10 18:05:53 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:299] - INFO: Processing audio with duration 00:29.024 2024-05-10 18:05:53 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【对不起,我无法回答这个问题。】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_5.wav 2024-05-10 18:05:53 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\Miniconda3\lib\site-packages\faster_whisper\transcribe.py[line:353] - INFO: Detected language 'ja' with probability 0.41 2024-05-10 18:05:53 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【您有其他问题需要我的帮助吗?】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_6.wav 有2段描述动作的语句 (1894年5月27日-1963年10月29日) 2024-05-10 18:05:54 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\webui.py[line:659] - INFO: callback接口 收到数据:{'type': 'llm', 'data': {'type': 'qwen', 'username': '主人', 'content_type': 'answer', 'content': '好的,我已经为您翻译了文本:Nicolai Winther 是一位挪威裔丹麦作家 、诗人和记者。他的作品包括小说、散文、诗歌和剧本等。他以独特的写作风格和深刻的主题而闻名,并在文学界享有很高的声誉。', 'timestamp': '2024-05-10 18:05:54'}} 2024-05-10 18:05:54 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\my_handle.py[line:2116] - INFO: [AI回复主人]:好的,我已经为您翻译了文本:Nicolai Winther 是一位挪威裔丹麦作家、诗人和记者。他的作品包括小说、散文、诗歌和剧本等。他以独特的写作风格和深刻的主题而闻名,并在文学界享有很高的声誉。 2024-05-10 18:05:54 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:54 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:54 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:423] - INFO: 优先级: 20 2024-05-10 18:05:54 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【好的,我已经为您翻译了文本:Nicolai Winther 是一位挪威裔丹麦作家、诗人和记者。】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_7.wav 2024-05-10 18:05:54 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【他的作品包括小说、散文、诗歌和剧本等。】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_8.wav 2024-05-10 18:05:55 - h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\utils\audio.py[line:1013] - INFO: bert_vits2合成成功,合成内容:【他以独特的写作风格和深刻的主题而闻名,并在文学界享有很高的声誉。】,输出到=h:\AIVtuber\share\AI Vtuber\AI-Vtuber-20240417\out\bert_vits2_9.wav

cracya01 commented 4 months ago

经排查确实是VB-voicemeeter的问题,换了 Virtual Audio Cable (VAC)之后就好了。