Closed White-Friday closed 7 months ago
Notice: In order to resolve issues more efficiently, please raise issue following the template. (注意:为了更加高效率解决您遇到的问题,请按照模板提问,补充细节)
在使用流式模型speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online,按照modelscope上面的代码示例,语音文件更换为自己的语音,流式识别的效果很差,几乎不能使用。使用非流式的语音识别模型,效果还是很好的
实际小部分输出效果,text基本为空: [{'key': 'rand_key_xVktYncJB4AIv', 'text': ''}] [{'key': 'rand_key_nisfidv7diPeH', 'text': ''}] [{'key': 'rand_key_hhUdhjy389Ntb', 'text': ''}] [{'key': 'rand_key_CgmMUMGJaZoka', 'text': ''}] [{'key': 'rand_key_KCq3s4MM19l62', 'text': ''}] [{'key': 'rand_key_GMz5ugkhJjEja', 'text': ''}] [{'key': 'rand_key_MctnhqqCF5Z7o', 'text': ''}] [{'key': 'rand_key_r4RMJB5Dt9CoM', 'text': ''}] [{'key': 'rand_key_IAgAzmhVx0OPP', 'text': ''}] [{'key': 'rand_key_Kp4Q3VbGRebTA', 'text': ''}] [{'key': 'rand_key_wmTnvBgc3U5FB', 'text': ''}] [{'key': 'rand_key_3try9M9ryLjOi', 'text': ''}] [{'key': 'rand_key_belcu3MxHD1BZ', 'text': ''}] [{'key': 'rand_key_mzdwXRGyQQDKM', 'text': '是'}] 下面是代码:
from funasr import AutoModel import soundfile import os chunk_size = [0, 10, 5] #[0, 10, 5] 600ms, [0, 8, 4] 480ms encoder_chunk_look_back = 4 #number of chunks to lookback for encoder self-attention decoder_chunk_look_back = 1 #number of encoder chunks to lookback for decoder cross-attention model = AutoModel( model="iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online", model_revision="v2.0.4" #model="iic/speech_paraformer-large_asr_nat-zh-cantonese-en-16k-vocab8501-online", model_revision="master" #vad_model="iic/speech_fsmn_vad_zh-cn-16k-common-pytorch", vad_model_revision="v2.0.4", #punc_model="iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch", punc_model_revision="v2.0.4" ) cache = {} wav_file = "wavs/test2.wav" speech, sample_rate = soundfile.read(wav_file) chunk_stride = chunk_size[1] * 960 # 600ms、480ms cache = {} total_chunk_num = int(len((speech)-1)/chunk_stride+1) for i in range(total_chunk_num): speech_chunk = speech[i*chunk_stride:(i+1)*chunk_stride] is_final = i == total_chunk_num - 1 res = model.generate(input=speech_chunk, cache=cache, is_final=is_final, chunk_size=chunk_size, encoder_chunk_look_back=encoder_chunk_look_back, decoder_chunk_look_back=decoder_chunk_look_back, ) print(res)
usesoxi to check the audio format.
soxi
Notice: In order to resolve issues more efficiently, please raise issue following the template. (注意:为了更加高效率解决您遇到的问题,请按照模板提问,补充细节)
❓ Questions and Help
在使用流式模型speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online,按照modelscope上面的代码示例,语音文件更换为自己的语音,流式识别的效果很差,几乎不能使用。使用非流式的语音识别模型,效果还是很好的
Before asking:
实际小部分输出效果,text基本为空: [{'key': 'rand_key_xVktYncJB4AIv', 'text': ''}] [{'key': 'rand_key_nisfidv7diPeH', 'text': ''}] [{'key': 'rand_key_hhUdhjy389Ntb', 'text': ''}] [{'key': 'rand_key_CgmMUMGJaZoka', 'text': ''}] [{'key': 'rand_key_KCq3s4MM19l62', 'text': ''}] [{'key': 'rand_key_GMz5ugkhJjEja', 'text': ''}] [{'key': 'rand_key_MctnhqqCF5Z7o', 'text': ''}] [{'key': 'rand_key_r4RMJB5Dt9CoM', 'text': ''}] [{'key': 'rand_key_IAgAzmhVx0OPP', 'text': ''}] [{'key': 'rand_key_Kp4Q3VbGRebTA', 'text': ''}] [{'key': 'rand_key_wmTnvBgc3U5FB', 'text': ''}] [{'key': 'rand_key_3try9M9ryLjOi', 'text': ''}] [{'key': 'rand_key_belcu3MxHD1BZ', 'text': ''}] [{'key': 'rand_key_mzdwXRGyQQDKM', 'text': '是'}] 下面是代码: