Fay is an open-source digital human framework integrating language models and digital characters. It offers retail, assistant, and agent versions for diverse applications like virtual shopping guides, broadcasters, assistants, waiters, teachers, and voice or text-based mobile assistants.
启动信息正常：
2023-08-07 17:49:23,813 - modelscope - INFO - PyTorch version 2.0.1 Found.
2023-08-07 17:49:23,813 - modelscope - INFO - Loading ast index from /Users/wujun/.cache/modelscope/ast_indexer
2023-08-07 17:49:23,835 - modelscope - INFO - Loading done! Current index file version is 1.8.1, with md5 13dd78b3f57d10df28aacaaf0c9ae045 and a total number of 893 components indexed
model loading
model loaded
Exception in thread Thread-1 (asr):
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
self.run()
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 975, in run
self._target(self._args, self._kwargs)
File "/Users/wujun/PycharmProjects/Fay/test/funasr/ASR_server.py", line 188, in asr
rec_result = inference_pipeline_asr(audio_in=audio_in)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/modelscope/pipelines/audio/asr_inference_pipeline.py", line 256, in call
output = self.forward(output, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/modelscope/pipelines/audio/asr_inference_pipeline.py", line 505, in forward
inputs['asr_result'] = self.run_inference(self.cmd, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/modelscope/pipelines/audio/asr_inference_pipeline.py", line 580, in run_inference
asr_result = self.funasr_infer_modelscope(cmd['name_and_type'],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/funasr/bin/asr_inference_launch.py", line 368, in _forward
results = speech2text(batch)
^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(args, kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/funasr/bin/asr_infer.py", line 452, in call
decoder_outs = self.asr_model.cal_decoder_with_predictor(enc,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/funasr/models/e2e_asr_paraformer.py", line 2128, in cal_decoder_with_predictor
hw_embed = torch.nn.utils.rnn.pack_padded_sequence(hw_embed, hw_lengths, batch_first=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/torch/nn/utils/rnn.py", line 263, in pack_padded_sequence
_VF._pack_padded_sequence(input, lengths, batch_first)
RuntimeError: Length of all samples has to be greater than 0, but found an element in 'lengths' that is <= 0**
xszyou
commented
9 months ago
首先更改了ASR相关的配置项：
funasr / ali
ASR_mode=funasr
ASR二选一（需要运行fay/test/funasr服务）集成达摩院asr项目、感谢中科大脑算法工程师张聪聪提供集成代码
local_asr_ip=0.0.0.0 local_asr_port=10197
funasr成功安装并且按照如下命令启动： python -u ASR_server.py --host "0.0.0.0" --port 10197 --ngpu 0 --model ./data/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404
启动信息正常： 2023-08-07 17:49:23,813 - modelscope - INFO - PyTorch version 2.0.1 Found. 2023-08-07 17:49:23,813 - modelscope - INFO - Loading ast index from /Users/wujun/.cache/modelscope/ast_indexer 2023-08-07 17:49:23,835 - modelscope - INFO - Loading done! Current index file version is 1.8.1, with md5 13dd78b3f57d10df28aacaaf0c9ae045 and a total number of 893 components indexed model loading model loaded
打开另外一个客户端，运行FAY主程序：python main.py, 对话时语音识别部分异常：
127.0.0.1 - - [2023-08-07 18:39:17] "POST /api/start-live HTTP/1.1" 200 245 1.065442 [18:39:23][系统] 聆听中... [18:39:24][系统] 语音处理中... [18:39:25][系统] 语音处理完成！ 耗时: 1011 ms [18:39:25][系统] [!] 语音未检测到内容！ [18:39:49][系统] 聆听中... [18:39:51][系统] 语音处理中... [18:39:52][系统] 语音处理完成！ 耗时: 1008 ms [18:39:52][系统] [!] 语音未检测到内容！
funasr服务报错：
Exception in thread Thread-1 (asr): Traceback (most recent call last): File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1038, in _bootstrap_inner self.run() File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 975, in run self._target(self._args, self._kwargs) File "/Users/wujun/PycharmProjects/Fay/test/funasr/ASR_server.py", line 188, in asr rec_result = inference_pipeline_asr(audio_in=audio_in) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/modelscope/pipelines/audio/asr_inference_pipeline.py", line 256, in call output = self.forward(output, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/modelscope/pipelines/audio/asr_inference_pipeline.py", line 505, in forward inputs['asr_result'] = self.run_inference(self.cmd, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/modelscope/pipelines/audio/asr_inference_pipeline.py", line 580, in run_inference asr_result = self.funasr_infer_modelscope(cmd['name_and_type'], ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/funasr/bin/asr_inference_launch.py", line 368, in _forward results = speech2text(batch) ^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/funasr/bin/asr_infer.py", line 452, in call decoder_outs = self.asr_model.cal_decoder_with_predictor(enc, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/funasr/models/e2e_asr_paraformer.py", line 2128, in cal_decoder_with_predictor hw_embed = torch.nn.utils.rnn.pack_padded_sequence(hw_embed, hw_lengths, batch_first=True, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/wujun/PycharmProjects/Fay/venv/lib/python3.11/site-packages/torch/nn/utils/rnn.py", line 263, in pack_padded_sequence _VF._pack_padded_sequence(input, lengths, batch_first) RuntimeError: Length of all samples has to be greater than 0, but found an element in 'lengths' that is <= 0**
有谁遇到这个情况吗？是什么原因呢？求指点～